[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:21 dist-test-slave-mvvj krb5kdc[24871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:21 dist-test-slave-mvvj krb5kdc[24871](info): set up 2 sockets
Aug 11 22:25:21 dist-test-slave-mvvj krb5kdc[24871](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 22:25:22.256233 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.009s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:22 dist-test-slave-mvvj krb5kdc[24871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:22.409134 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38251
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:38251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:22.537858 24886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:22.538239 24886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:22.538352 24886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:22.542553 24886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:22.542665 24886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:22.542702 24886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:22.542743 24886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:22.542778 24886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:22.548732 24886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:38251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:22.550356 24886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:22.551790 24886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:22.564199 24886 server_base.cc:1034] running on GCE node
W20260811 22:25:22.564214 24894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:22.564227 24891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:22.564729 24892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:22.565496 24886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:22.567381 24886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:22.568660 24886 hybrid_clock.cc:648] HybridClock initialized: now 1786487122568615 us; error 84 us; skew 500 ppm
Aug 11 22:25:22 dist-test-slave-mvvj krb5kdc[24871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487122, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:22.572094 24886 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:22.573396 24886 webserver.cc:460] Webserver started at http://127.24.71.254:43439/ using document root <none> and password file <none>
I20260811 22:25:22.574060 24886 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:22.574129 24886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:22.574455 24886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:22.576797 24886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "88e18657227747f1a0bb0ca597f20a30"
format_stamp: "Formatted at 2026-08-11 22:25:22 on dist-test-slave-mvvj"
server_key: "a93a0415c240f23cdf47209ee7fd00f1"
server_key_iv: "f3361d7f93aa42d036c8d7e9a9ebb54d"
server_key_version: "encryptionkey@0"
I20260811 22:25:22.577396 24886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "88e18657227747f1a0bb0ca597f20a30"
format_stamp: "Formatted at 2026-08-11 22:25:22 on dist-test-slave-mvvj"
server_key: "a93a0415c240f23cdf47209ee7fd00f1"
server_key_iv: "f3361d7f93aa42d036c8d7e9a9ebb54d"
server_key_version: "encryptionkey@0"
I20260811 22:25:22.581840 24886 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:25:22.584767 24903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:22.586371 24886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:25:22.586624 24886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "88e18657227747f1a0bb0ca597f20a30"
format_stamp: "Formatted at 2026-08-11 22:25:22 on dist-test-slave-mvvj"
server_key: "a93a0415c240f23cdf47209ee7fd00f1"
server_key_iv: "f3361d7f93aa42d036c8d7e9a9ebb54d"
server_key_version: "encryptionkey@0"
I20260811 22:25:22.586760 24886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:22.600858 24886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:22.605237 24886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:22.605489 24886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:22.618968 24886 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:38251
I20260811 22:25:22.619001 24961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:38251 every 8 connection(s)
I20260811 22:25:22.620404 24886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:22.624958 24962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:22.628710 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 24886
I20260811 22:25:22.628867 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:22.630513 24863 external_mini_cluster.cc:1428] Setting key 83102e3fe86ad816f56d0ab4cdd72adb
I20260811 22:25:22.632922 24962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30: Bootstrap starting.
I20260811 22:25:22.636226 24962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:22.637392 24962 log.cc:824] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:22.640867 24962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30: No bootstrap required, opened a new log
I20260811 22:25:22.647435 24962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e18657227747f1a0bb0ca597f20a30" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38251 } }
I20260811 22:25:22.647898 24962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:22.647984 24962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e18657227747f1a0bb0ca597f20a30, State: Initialized, Role: FOLLOWER
I20260811 22:25:22.648829 24962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e18657227747f1a0bb0ca597f20a30" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38251 } }
I20260811 22:25:22.649111 24962 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:22.649240 24962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:22.649391 24962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:22.650877 24962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e18657227747f1a0bb0ca597f20a30" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38251 } }
I20260811 22:25:22.651412 24962 leader_election.cc:302] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88e18657227747f1a0bb0ca597f20a30; no voters:
I20260811 22:25:22.651940 24962 leader_election.cc:288] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:22.652110 24968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:22.652377 24968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [term 1 LEADER]: Becoming Leader. State: Replica: 88e18657227747f1a0bb0ca597f20a30, State: Running, Role: LEADER
I20260811 22:25:22.652817 24968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e18657227747f1a0bb0ca597f20a30" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38251 } }
I20260811 22:25:22.654078 24962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:22.655642 24970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88e18657227747f1a0bb0ca597f20a30. Latest consensus state: current_term: 1 leader_uuid: "88e18657227747f1a0bb0ca597f20a30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e18657227747f1a0bb0ca597f20a30" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38251 } } }
I20260811 22:25:22.655810 24970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:22.656277 24969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88e18657227747f1a0bb0ca597f20a30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e18657227747f1a0bb0ca597f20a30" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38251 } } }
I20260811 22:25:22.656399 24969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:22.656802 24977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 22:25:22 dist-test-slave-mvvj krb5kdc[24871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:22.661234 24977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:22.667825 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:22.635574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.24.71.254:38251)
0811 22:25:22.636270 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:22.636296 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:22.638795 (+ 2499us) server_negotiation.cc:407] Connection header received
0811 22:25:22.644260 (+ 5465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:22.644280 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:22.644729 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:22.645036 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:22.649225 (+ 4189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:22.650896 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:22.652819 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:22.653619 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:22.653829 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:22.660273 (+ 6444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:22.660281 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:22.660299 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:22.660342 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:22.662864 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:22.663786 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:22.663793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:22.663799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:22.663995 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:22.664493 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:22.664496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:22.664498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:22.665191 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:22.665400 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:22.666272 (+ 872us) server_negotiation.cc:299] Negotiation successful
0811 22:25:22.666692 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":210,"threads_started":1}
I20260811 22:25:22.669544 24977 catalog_manager.cc:1347] Generated new cluster ID: 83568cc24d134e58a274381252573961
I20260811 22:25:22.669698 24977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:22.701375 24977 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:22.702606 24977 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:22.711151 24977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88e18657227747f1a0bb0ca597f20a30: Generated new TSK 0
I20260811 22:25:22.712304 24977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:25:22.764302 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38251
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:22.903381 24991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:22.903683 24991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:22.903730 24991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:22.908344 24991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:22.908432 24991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:22.908495 24991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:22.914059 24991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38251
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:22.915246 24991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:22.916335 24991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:22.928083 24991 server_base.cc:1034] running on GCE node
W20260811 22:25:22.928040 24997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:22.928109 24996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:22.928109 25001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:22.928891 24991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:22.929644 24991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:22.930862 24991 hybrid_clock.cc:648] HybridClock initialized: now 1786487122930822 us; error 53 us; skew 500 ppm
Aug 11 22:25:22 dist-test-slave-mvvj krb5kdc[24871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487122, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:22.934244 24991 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:25:22.935413 24991 webserver.cc:460] Webserver started at http://127.24.71.193:45821/ using document root <none> and password file <none>
I20260811 22:25:22.936124 24991 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:22.936195 24991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:22.936436 24991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:22.939051 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "73e1a233f6de4346ae95db9f2e6dc01e"
format_stamp: "Formatted at 2026-08-11 22:25:22 on dist-test-slave-mvvj"
server_key: "8fbb12ab9d8f0d73847d4f599247b1c6"
server_key_iv: "e04279eb0a540bad6e04f442745f1675"
server_key_version: "encryptionkey@0"
I20260811 22:25:22.939745 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "73e1a233f6de4346ae95db9f2e6dc01e"
format_stamp: "Formatted at 2026-08-11 22:25:22 on dist-test-slave-mvvj"
server_key: "8fbb12ab9d8f0d73847d4f599247b1c6"
server_key_iv: "e04279eb0a540bad6e04f442745f1675"
server_key_version: "encryptionkey@0"
I20260811 22:25:22.944347 24991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:25:22.947117 25010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:22.948649 24991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:22.948808 24991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "73e1a233f6de4346ae95db9f2e6dc01e"
format_stamp: "Formatted at 2026-08-11 22:25:22 on dist-test-slave-mvvj"
server_key: "8fbb12ab9d8f0d73847d4f599247b1c6"
server_key_iv: "e04279eb0a540bad6e04f442745f1675"
server_key_version: "encryptionkey@0"
I20260811 22:25:22.948944 24991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:22.966296 24991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:22.970117 24991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:22.970378 24991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:22.971163 24991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:22.972380 24991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:22.972436 24991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:22.972478 24991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:22.972541 24991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:22.983453 24991 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:45641
I20260811 22:25:22.983484 25134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:45641 every 8 connection(s)
I20260811 22:25:22.984573 24991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:22.993788 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 24991
I20260811 22:25:22.993947 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:22.994278 24863 external_mini_cluster.cc:1428] Setting key a5913881b7a52759ae576573b86d9bec
Aug 11 22:25:22 dist-test-slave-mvvj krb5kdc[24871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487122, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:23.002365 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:22.987095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:36459 (local address 127.24.71.254:38251)
0811 22:25:22.987286 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:22.987291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:22.988207 (+ 916us) server_negotiation.cc:407] Connection header received
0811 22:25:22.989781 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:22.989784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:22.989837 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:22.989950 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:22.991387 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:22.992089 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:22.993436 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:22.993755 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:22.993866 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:22.997090 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:22.997098 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:22.997105 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:22.997148 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:22.998874 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:22.999635 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:22.999639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:22.999640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:22.999773 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.000288 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.000310 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.000312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.000580 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:23.000688 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.001831 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.002085 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:25:23.003181 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:22.987483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38251 (local address 127.24.71.193:36459)
0811 22:25:22.988042 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:22.988089 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:22.989562 (+ 1473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:22.990128 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:22.990135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:22.990605 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:22.991174 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:22.991193 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:22.992242 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:22.992245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:22.993266 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:22.993277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:22.993974 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:22.993978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:22.994209 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:22.995127 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:22.995147 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:22.996878 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:22.999025 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:22.999036 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:22.999061 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:22.999501 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:22.999939 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:22.999943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:22.999947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.000180 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.000756 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:23.000764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:23.001297 (+ 533us) client_negotiation.cc:769] Sending connection context
0811 22:25:23.001732 (+ 435us) client_negotiation.cc:240] Negotiation successful
0811 22:25:23.002081 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":132,"threads_started":1}
I20260811 22:25:23.004923 25135 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38251
I20260811 22:25:23.005218 25135 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:23.005980 25135 heartbeater.cc:511] Master 127.24.71.254:38251 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:25:23.008656 24921 ts_manager.cc:194] Registered new tserver with Master: 73e1a233f6de4346ae95db9f2e6dc01e (127.24.71.193:45641)
I20260811 22:25:23.010617 24921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:36459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:25:23.032251 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38251
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:23.156097 25142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:23.156443 25142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:23.156612 25142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:23.161121 25142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:23.161226 25142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:23.161324 25142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:23.167188 25142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38251
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:23.168558 25142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:23.169704 25142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:23.180613 25142 server_base.cc:1034] running on GCE node
W20260811 22:25:23.180460 25149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:23.180609 25150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:23.180460 25153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:23.181380 25142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:23.182158 25142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:23.183353 25142 hybrid_clock.cc:648] HybridClock initialized: now 1786487123183310 us; error 63 us; skew 500 ppm
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[24871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:23.186661 25142 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:25:23.187752 25142 webserver.cc:460] Webserver started at http://127.24.71.194:36079/ using document root <none> and password file <none>
I20260811 22:25:23.188475 25142 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:23.188551 25142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:23.188751 25142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:23.191191 25142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "99b03079209f45bb8600532cf1cac859"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-mvvj"
server_key: "821b0ca413471c37fbfdcf15fa6554a5"
server_key_iv: "87e33e3025b1602971019f66c56bb3a0"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.192025 25142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "99b03079209f45bb8600532cf1cac859"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-mvvj"
server_key: "821b0ca413471c37fbfdcf15fa6554a5"
server_key_iv: "87e33e3025b1602971019f66c56bb3a0"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.196663 25142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:23.199942 25161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.201632 25142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:25:23.201805 25142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "99b03079209f45bb8600532cf1cac859"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-mvvj"
server_key: "821b0ca413471c37fbfdcf15fa6554a5"
server_key_iv: "87e33e3025b1602971019f66c56bb3a0"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.201952 25142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:23.217618 25142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:23.221351 25142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:23.221601 25142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:23.222296 25142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:23.223232 25142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:23.223282 25142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.223351 25142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:23.223384 25142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.234007 25142 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:46673
I20260811 22:25:23.234031 25288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:46673 every 8 connection(s)
I20260811 22:25:23.235069 25142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:23.240506 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 25142
I20260811 22:25:23.240612 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:23.240916 24863 external_mini_cluster.cc:1428] Setting key a831268e396d361dd1d7e53fd04f7e8f
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[24871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:25:23.253541 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.237409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:47473 (local address 127.24.71.254:38251)
0811 22:25:23.237602 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.237605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.238510 (+ 905us) server_negotiation.cc:407] Connection header received
0811 22:25:23.240014 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.240018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.240086 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.240195 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:23.241769 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.242523 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.244763 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.245070 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.245222 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.248679 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:23.248684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:23.248686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:23.248719 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:23.250551 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.251393 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.251397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.251399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.251496 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.252045 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.252048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.252049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.252281 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:23.252379 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.253222 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.253391 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 22:25:23.255152 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.237763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38251 (local address 127.24.71.194:47473)
0811 22:25:23.238277 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:23.238338 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:23.239731 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:23.240413 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:23.240420 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:23.240906 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:23.241610 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.241631 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.242734 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.242738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.244568 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.244588 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.245304 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.245314 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.245491 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.246250 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:23.246269 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:23.248396 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:23.250712 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.250721 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.250744 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.251167 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.251616 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.251622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.251626 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.251885 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.252420 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:23.252430 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:23.252867 (+ 437us) client_negotiation.cc:769] Sending connection context
0811 22:25:23.253768 (+ 901us) client_negotiation.cc:240] Negotiation successful
0811 22:25:23.254142 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":140,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:25:23.256932 25289 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38251
I20260811 22:25:23.257236 25289 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:23.258008 25289 heartbeater.cc:511] Master 127.24.71.254:38251 requested a full tablet report, sending...
I20260811 22:25:23.259411 24921 ts_manager.cc:194] Registered new tserver with Master: 99b03079209f45bb8600532cf1cac859 (127.24.71.194:46673)
I20260811 22:25:23.260133 24921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:47473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:25:23.279112 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38251
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:23.402410 25294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:23.402760 25294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:23.402844 25294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:23.407361 25294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:23.407454 25294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:23.407550 25294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:23.413264 25294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38251
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:23.414543 25294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:23.415705 25294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:23.426265 25305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:23.426265 25300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:23.426265 25301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:23.426690 25294 server_base.cc:1034] running on GCE node
I20260811 22:25:23.427158 25294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:23.428041 25294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:23.429248 25294 hybrid_clock.cc:648] HybridClock initialized: now 1786487123429207 us; error 52 us; skew 500 ppm
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[24871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:23.432472 25294 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:25:23.433528 25294 webserver.cc:460] Webserver started at http://127.24.71.195:38381/ using document root <none> and password file <none>
I20260811 22:25:23.434171 25294 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:23.434271 25294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:23.434526 25294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:23.436928 25294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "48c098674e8e4dfcb2cc5087042f2328"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-mvvj"
server_key: "a946d3e6108d60eb80c4762d5c69f321"
server_key_iv: "d6142f1f7b78d7847ef09f6af9f592ff"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.437811 25294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "48c098674e8e4dfcb2cc5087042f2328"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-mvvj"
server_key: "a946d3e6108d60eb80c4762d5c69f321"
server_key_iv: "d6142f1f7b78d7847ef09f6af9f592ff"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.442734 25294 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:25:23.445591 25314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.447150 25294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:23.447310 25294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "48c098674e8e4dfcb2cc5087042f2328"
format_stamp: "Formatted at 2026-08-11 22:25:23 on dist-test-slave-mvvj"
server_key: "a946d3e6108d60eb80c4762d5c69f321"
server_key_iv: "d6142f1f7b78d7847ef09f6af9f592ff"
server_key_version: "encryptionkey@0"
I20260811 22:25:23.447448 25294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:23.474216 25294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:23.478048 25294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:23.478241 25294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:23.478854 25294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:23.479785 25294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:23.479888 25294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.479962 25294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:23.480016 25294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:23.489648 25294 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:38177
I20260811 22:25:23.489697 25445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:38177 every 8 connection(s)
I20260811 22:25:23.490741 25294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:23.497622 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 25294
I20260811 22:25:23.497741 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:23.498085 24863 external_mini_cluster.cc:1428] Setting key 836cf9cc3aa74ac1aaee5c077643d90b
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[24871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:23.506745 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.493048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:44673 (local address 127.24.71.254:38251)
0811 22:25:23.493337 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.493343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.494192 (+ 849us) server_negotiation.cc:407] Connection header received
0811 22:25:23.495738 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.495741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.495796 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.495940 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:23.497271 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.497980 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.499668 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.500056 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.500176 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.502933 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:23.502936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:23.502939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:23.502965 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:23.504577 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.505119 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.505121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.505123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.505200 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.505554 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.505556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.505557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.505750 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:23.505820 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.506459 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.506596 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260811 22:25:23.507916 25449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.493504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38251 (local address 127.24.71.195:44673)
0811 22:25:23.494025 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:23.494071 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:23.495504 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:23.496129 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:23.496136 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:23.496564 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:23.497091 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.497104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.498132 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.498137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.499526 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:23.499538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.500221 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.500225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:23.500404 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.501126 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:23.501150 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:23.502762 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:23.504693 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.504699 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.504714 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.505020 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.505292 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:23.505294 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:23.505295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:23.505470 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:23.505854 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:23.505861 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:23.506157 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 22:25:23.506366 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 22:25:23.506809 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":135,"threads_started":1}
I20260811 22:25:23.509167 25446 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38251
I20260811 22:25:23.509451 25446 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:23.510010 25446 heartbeater.cc:511] Master 127.24.71.254:38251 requested a full tablet report, sending...
I20260811 22:25:23.511186 24921 ts_manager.cc:194] Registered new tserver with Master: 48c098674e8e4dfcb2cc5087042f2328 (127.24.71.195:38177)
I20260811 22:25:23.511755 24921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:44673
I20260811 22:25:23.513365 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[24871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[24871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:25:23.542733 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.532403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40900 (local address 127.24.71.193:45641)
0811 22:25:23.532736 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.532746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.532775 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:25:23.532970 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.532976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.533185 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.533343 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:23.534087 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.534848 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.535708 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.536186 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.536307 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.539008 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:23.539020 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:23.539036 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:23.539064 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:23.540797 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.541282 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.541289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.541290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.541375 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.541672 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.541676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.541678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.541976 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:23.542204 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.542391 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.542552 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":98,"threads_started":1}
W20260811 22:25:23.543939 25042 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40900
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[24871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[24871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:25:23.574267 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:23.565050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.24.71.193:45641)
0811 22:25:23.565239 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:23.565244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:23.565402 (+ 158us) server_negotiation.cc:407] Connection header received
0811 22:25:23.565583 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:23.565587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:23.565639 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:23.565737 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:23.566633 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.567153 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.568112 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:23.568341 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:23.568458 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:23.570990 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:23.570995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:23.570998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:23.571031 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:23.572467 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.572957 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.572961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.572962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.573041 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:23.573334 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:23.573338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:23.573339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:23.573543 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:23.573693 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:23.573958 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 22:25:23.574089 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:25:23.575836 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 24991
I20260811 22:25:23.585217 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 25142
I20260811 22:25:23.594756 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 25294
I20260811 22:25:23.602816 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 24886
2026-08-11T22:25:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2491 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[25465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[25465](info): set up 2 sockets
Aug 11 22:25:23 dist-test-slave-mvvj krb5kdc[25465](info): commencing operation
krb5kdc: starting...
W20260811 22:25:24.811972 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:24 dist-test-slave-mvvj krb5kdc[25465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:24.961009 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:41545
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:46641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:41545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:25.089331 25480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:25.089679 25480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:25.089771 25480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:25.094062 25480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:25.094169 25480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:25.094211 25480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:25.094272 25480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:25.094310 25480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:25.100121 25480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:46641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:41545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:25.101368 25480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:25.102509 25480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:25.111975 25490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.111923 25486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.112020 25487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:25.112105 25480 server_base.cc:1034] running on GCE node
I20260811 22:25:25.112859 25480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:25.114063 25480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:25.115279 25480 hybrid_clock.cc:648] HybridClock initialized: now 1786487125115236 us; error 57 us; skew 500 ppm
Aug 11 22:25:25 dist-test-slave-mvvj krb5kdc[25465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:25.118396 25480 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:25.119472 25480 webserver.cc:460] Webserver started at http://127.24.71.254:45127/ using document root <none> and password file <none>
I20260811 22:25:25.120189 25480 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:25.120291 25480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:25.120549 25480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:25.122792 25480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "3f9ca283f52f4c7eadc5354a4337dd0e"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "b958fabad30a6a6727ef1637bf61ab66"
server_key_iv: "a60d979f25b1d6cfd02e0ca430f974eb"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.123426 25480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "3f9ca283f52f4c7eadc5354a4337dd0e"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "b958fabad30a6a6727ef1637bf61ab66"
server_key_iv: "a60d979f25b1d6cfd02e0ca430f974eb"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.127688 25480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:25:25.130750 25501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.132359 25480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:25.132546 25480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "3f9ca283f52f4c7eadc5354a4337dd0e"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "b958fabad30a6a6727ef1637bf61ab66"
server_key_iv: "a60d979f25b1d6cfd02e0ca430f974eb"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.132699 25480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:25.160218 25480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:25.164247 25480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:25.164541 25480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:25.173393 25480 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:41545
I20260811 22:25:25.173403 25557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:41545 every 8 connection(s)
I20260811 22:25:25.174582 25480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:25.177935 25558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:25.180339 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 25480
I20260811 22:25:25.180506 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:25.180784 24863 external_mini_cluster.cc:1428] Setting key 9372d090f920404d0dc53c1d954b814c
I20260811 22:25:25.184854 25558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e: Bootstrap starting.
I20260811 22:25:25.187757 25558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:25.188779 25558 log.cc:824] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:25:25 dist-test-slave-mvvj krb5kdc[25465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:25.191521 25558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e: No bootstrap required, opened a new log
I20260811 22:25:25.196182 25558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f9ca283f52f4c7eadc5354a4337dd0e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41545 } }
I20260811 22:25:25.196460 25558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:25.196499 25558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9ca283f52f4c7eadc5354a4337dd0e, State: Initialized, Role: FOLLOWER
I20260811 22:25:25.197037 25558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f9ca283f52f4c7eadc5354a4337dd0e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41545 } }
I20260811 22:25:25.197167 25558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:25.197250 25558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:25.197273 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.182473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.24.71.254:41545)
0811 22:25:25.182957 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.182976 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.183016 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:25:25.183745 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.183767 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.184209 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.184533 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:25.185538 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.186588 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.187795 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.188371 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.188536 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.191596 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:25.191605 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:25.191627 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:25.191671 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:25.193647 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:25.194282 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:25.194288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:25.194296 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:25.194477 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:25.194834 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:25.194837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:25.194838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:25.195322 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:25.195505 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.195895 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.196282 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":177,"threads_started":1}
I20260811 22:25:25.197365 25558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:25.198449 25558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f9ca283f52f4c7eadc5354a4337dd0e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41545 } }
I20260811 22:25:25.198853 25558 leader_election.cc:302] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f9ca283f52f4c7eadc5354a4337dd0e; no voters:
I20260811 22:25:25.199247 25558 leader_election.cc:288] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:25.199445 25563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:25.199774 25563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [term 1 LEADER]: Becoming Leader. State: Replica: 3f9ca283f52f4c7eadc5354a4337dd0e, State: Running, Role: LEADER
I20260811 22:25:25.200172 25563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f9ca283f52f4c7eadc5354a4337dd0e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41545 } }
I20260811 22:25:25.200614 25558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:25.201462 25563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f9ca283f52f4c7eadc5354a4337dd0e. Latest consensus state: current_term: 1 leader_uuid: "3f9ca283f52f4c7eadc5354a4337dd0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f9ca283f52f4c7eadc5354a4337dd0e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41545 } } }
I20260811 22:25:25.201592 25563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:25.202090 25564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f9ca283f52f4c7eadc5354a4337dd0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f9ca283f52f4c7eadc5354a4337dd0e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41545 } } }
I20260811 22:25:25.202193 25564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:25.202378 25575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:25.205876 25575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:25.212445 25575 catalog_manager.cc:1347] Generated new cluster ID: 3a156c9fefd04464a637c270e4306cfe
I20260811 22:25:25.212555 25575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:25.230242 25575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:25.231312 25575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:25.236270 25575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f9ca283f52f4c7eadc5354a4337dd0e: Generated new TSK 0
I20260811 22:25:25.237149 25575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:25:25.276207 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:41545
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:46641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:25.408574 25588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:25.408946 25588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:25.409004 25588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:25.413519 25588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:25.413611 25588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:25.413677 25588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:25.422271 25588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:46641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:41545
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:25.424103 25588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:25.425582 25588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:25.437203 25594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.437186 25597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.437206 25595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:25.437539 25588 server_base.cc:1034] running on GCE node
I20260811 22:25:25.438129 25588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:25.438882 25588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:25.440092 25588 hybrid_clock.cc:648] HybridClock initialized: now 1786487125440076 us; error 28 us; skew 500 ppm
Aug 11 22:25:25 dist-test-slave-mvvj krb5kdc[25465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:25.455539 25588 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:25:25.457108 25588 webserver.cc:460] Webserver started at http://127.24.71.193:35865/ using document root <none> and password file <none>
I20260811 22:25:25.457973 25588 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:25.458081 25588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:25.458398 25588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:25.461854 25588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "2ef6df919b4d49f18ba95f2f837a5add"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "75dae66f1efdd979a0e9867d9b300be9"
server_key_iv: "a2db715020d571a4b01bcf24e33d655c"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.462662 25588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "2ef6df919b4d49f18ba95f2f837a5add"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "75dae66f1efdd979a0e9867d9b300be9"
server_key_iv: "a2db715020d571a4b01bcf24e33d655c"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.468020 25588 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 22:25:25.470917 25604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.472620 25588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:25.472755 25588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "2ef6df919b4d49f18ba95f2f837a5add"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "75dae66f1efdd979a0e9867d9b300be9"
server_key_iv: "a2db715020d571a4b01bcf24e33d655c"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.472864 25588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:25.493830 25588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:25.497435 25588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:25.497630 25588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:25.498255 25588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:25.499130 25588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:25.499181 25588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.499224 25588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:25.499239 25588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.509406 25588 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:39383
I20260811 22:25:25.509428 25735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:39383 every 8 connection(s)
I20260811 22:25:25.510483 25588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:25.520725 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 25588
I20260811 22:25:25.520867 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:25.521232 24863 external_mini_cluster.cc:1428] Setting key 5ff0cc4534d7f3538ac3ac57b11a21c3
Aug 11 22:25:25 dist-test-slave-mvvj krb5kdc[25465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:25.527695 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.512942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:41985 (local address 127.24.71.254:41545)
0811 22:25:25.513115 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.513119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.514536 (+ 1417us) server_negotiation.cc:407] Connection header received
0811 22:25:25.515437 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.515440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.515496 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.515594 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:25.517153 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.517718 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.519010 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.519266 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.519365 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.522419 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:25.522422 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:25.522424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:25.522453 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:25.524707 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:25.525300 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:25.525307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:25.525313 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:25.525439 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:25.525990 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:25.525994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:25.525995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:25.526205 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:25.526295 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.527206 (+ 911us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.527379 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:25:25.528566 25738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.513265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:41545 (local address 127.24.71.193:41985)
0811 22:25:25.514204 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:25.514251 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:25.515187 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:25.515793 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:25.515800 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:25.516381 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:25.516976 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.516992 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.517874 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.517878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.518858 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.518871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.519461 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.519464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.519625 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.520336 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:25.520353 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:25.522155 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:25.524866 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:25.524871 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:25.524889 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:25.525183 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:25.525604 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:25.525610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:25.525614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:25.525858 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:25.526326 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:25.526335 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:25.526813 (+ 478us) client_negotiation.cc:769] Sending connection context
0811 22:25:25.527079 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 22:25:25.527468 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":148,"threads_started":1}
I20260811 22:25:25.529995 25736 heartbeater.cc:347] Connected to a master server at 127.24.71.254:41545
I20260811 22:25:25.530319 25736 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:25.530893 25736 heartbeater.cc:511] Master 127.24.71.254:41545 requested a full tablet report, sending...
I20260811 22:25:25.532735 25519 ts_manager.cc:194] Registered new tserver with Master: 2ef6df919b4d49f18ba95f2f837a5add (127.24.71.193:39383)
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:25:25.533981 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:41985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:25:25.559686 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:41545
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:46641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:25.681639 25741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:25.682024 25741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:25.682109 25741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:25.686404 25741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:25.686539 25741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:25.686642 25741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:25.692636 25741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:46641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:41545
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:25.693929 25741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:25.695108 25741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:25.706786 25746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.706756 25750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.706774 25748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:25.707634 25741 server_base.cc:1034] running on GCE node
I20260811 22:25:25.708297 25741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:25.709134 25741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:25.710355 25741 hybrid_clock.cc:648] HybridClock initialized: now 1786487125710240 us; error 128 us; skew 500 ppm
Aug 11 22:25:25 dist-test-slave-mvvj krb5kdc[25465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:25.713759 25741 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:25:25.714820 25741 webserver.cc:460] Webserver started at http://127.24.71.194:46843/ using document root <none> and password file <none>
I20260811 22:25:25.715472 25741 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:25.715574 25741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:25.715876 25741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:25.718322 25741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "b576a59c9efe4beab36263cb86825766"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "425fb9190d48dcce753021b3980c2f22"
server_key_iv: "ac58a813234dcd446821fa0710a054bf"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.719012 25741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "b576a59c9efe4beab36263cb86825766"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "425fb9190d48dcce753021b3980c2f22"
server_key_iv: "ac58a813234dcd446821fa0710a054bf"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.723547 25741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:25.726697 25758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.728505 25741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:25.728691 25741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "b576a59c9efe4beab36263cb86825766"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "425fb9190d48dcce753021b3980c2f22"
server_key_iv: "ac58a813234dcd446821fa0710a054bf"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.728844 25741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:25.758786 25741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:25.762791 25741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:25.763032 25741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:25.763748 25741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:25.764729 25741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:25.764804 25741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.764876 25741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:25.764916 25741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.774911 25741 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:34073
I20260811 22:25:25.774959 25885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:34073 every 8 connection(s)
I20260811 22:25:25.776085 25741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:25.779280 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 25741
I20260811 22:25:25.779407 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:25.779713 24863 external_mini_cluster.cc:1428] Setting key 687593332762f6e45f1a0b99b2260508
Aug 11 22:25:25 dist-test-slave-mvvj krb5kdc[25465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:25:25.793176 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.778501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:58355 (local address 127.24.71.254:41545)
0811 22:25:25.778688 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:25.778692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:25.780031 (+ 1339us) server_negotiation.cc:407] Connection header received
0811 22:25:25.781812 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:25.781816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:25.781874 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:25.781956 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:25.783273 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.784186 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.785443 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.785852 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.785985 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.788984 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:25.788991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:25.788997 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:25.789040 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:25.790807 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:25.791390 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:25.791396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:25.791401 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:25.791528 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:25.791947 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:25.791950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:25.791951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:25.792164 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:25.792253 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:25.792913 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 22:25:25.793039 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:25:25.794282 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:25.779009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:41545 (local address 127.24.71.194:58355)
0811 22:25:25.779801 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:25.779896 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:25.781583 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:25.782158 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:25.782166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:25.782573 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:25.783110 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.783125 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.784344 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.784347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.785304 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:25.785315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:25.786025 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:25.786028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:25.786185 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:25.786881 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:25.786899 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:25.788776 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:25.790959 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:25.790965 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:25.790981 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:25.791270 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:25.791647 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:25.791649 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:25.791651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:25.791802 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:25.792266 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:25.792273 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:25.792558 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 22:25:25.792840 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 22:25:25.793174 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":202,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:25:25.795742 25886 heartbeater.cc:347] Connected to a master server at 127.24.71.254:41545
I20260811 22:25:25.796074 25886 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:25.796691 25886 heartbeater.cc:511] Master 127.24.71.254:41545 requested a full tablet report, sending...
I20260811 22:25:25.798051 25519 ts_manager.cc:194] Registered new tserver with Master: b576a59c9efe4beab36263cb86825766 (127.24.71.194:34073)
I20260811 22:25:25.798741 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:58355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:25:25.817891 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:41545
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:46641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:25.940886 25891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:25.941195 25891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:25.941246 25891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:25.945614 25891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:25.945690 25891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:25.945755 25891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:25.951627 25891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:46641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:41545
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:25.952911 25891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:25.954054 25891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:25.965378 25899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.965378 25896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:25.965421 25897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:25.965998 25891 server_base.cc:1034] running on GCE node
I20260811 22:25:25.966681 25891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:25.967494 25891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:25.968721 25891 hybrid_clock.cc:648] HybridClock initialized: now 1786487125968674 us; error 61 us; skew 500 ppm
Aug 11 22:25:25 dist-test-slave-mvvj krb5kdc[25465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:25.972158 25891 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:25:25.973470 25891 webserver.cc:460] Webserver started at http://127.24.71.195:46091/ using document root <none> and password file <none>
I20260811 22:25:25.974263 25891 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:25.974381 25891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:25.974663 25891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:25.977344 25891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "50bcb2985f6345939ea59304e54472e3"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "ff9b7b37369711a274ca2cea147fee63"
server_key_iv: "90535a7d4b6b9608be43414ade5dcb23"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.978024 25891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "50bcb2985f6345939ea59304e54472e3"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "ff9b7b37369711a274ca2cea147fee63"
server_key_iv: "90535a7d4b6b9608be43414ade5dcb23"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.982439 25891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:25.985466 25906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:25.987092 25891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:25.987258 25891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "50bcb2985f6345939ea59304e54472e3"
format_stamp: "Formatted at 2026-08-11 22:25:25 on dist-test-slave-mvvj"
server_key: "ff9b7b37369711a274ca2cea147fee63"
server_key_iv: "90535a7d4b6b9608be43414ade5dcb23"
server_key_version: "encryptionkey@0"
I20260811 22:25:25.987399 25891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:26.003777 25891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:26.008787 25891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:26.009075 25891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:26.009828 25891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:26.010800 25891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:26.010852 25891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:26.010895 25891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:26.010910 25891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:26.022339 25891 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:33087
I20260811 22:25:26.022357 26033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:33087 every 8 connection(s)
I20260811 22:25:26.023524 25891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:26.026872 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 25891
I20260811 22:25:26.027019 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:26.027361 24863 external_mini_cluster.cc:1428] Setting key d5b1511d1cbd3b885ee006c03e55c449
Aug 11 22:25:26 dist-test-slave-mvvj krb5kdc[25465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:26.040295 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.025698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:57425 (local address 127.24.71.254:41545)
0811 22:25:26.025852 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.025857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.027281 (+ 1424us) server_negotiation.cc:407] Connection header received
0811 22:25:26.028428 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.028432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.028493 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.028606 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:26.030135 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.030727 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.032014 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.032239 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.032334 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.035422 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:26.035426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:26.035428 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:26.035458 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:26.037690 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.038318 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.038321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.038322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.038412 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.038855 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.038859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.038861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.039087 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:26.039181 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.039954 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.040119 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:25:26.041211 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.026069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:41545 (local address 127.24.71.195:57425)
0811 22:25:26.027071 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:26.027139 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:26.028191 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:26.028817 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:26.028828 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:26.029342 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:26.029985 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.030004 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.030856 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.030858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.031776 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.031785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.032443 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.032447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.032619 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.033266 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:26.033285 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:26.035192 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:26.037839 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:26.037845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:26.037862 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:26.038171 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:26.038531 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:26.038534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:26.038535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:26.038712 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:26.039229 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:26.039235 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:26.039542 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 22:25:26.039734 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 22:25:26.040125 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":141,"threads_started":1}
I20260811 22:25:26.042640 26034 heartbeater.cc:347] Connected to a master server at 127.24.71.254:41545
I20260811 22:25:26.042902 26034 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:26.043424 26034 heartbeater.cc:511] Master 127.24.71.254:41545 requested a full tablet report, sending...
I20260811 22:25:26.044649 25519 ts_manager.cc:194] Registered new tserver with Master: 50bcb2985f6345939ea59304e54472e3 (127.24.71.195:33087)
I20260811 22:25:26.045307 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:57425
I20260811 22:25:26.055438 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:26.069218 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.061117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.24.71.254:41545)
0811 22:25:26.061317 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.061321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.061456 (+ 135us) server_negotiation.cc:407] Connection header received
0811 22:25:26.061543 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.061546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.061597 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.061689 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:26.062471 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.063022 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.063884 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.064134 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.064237 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.065775 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:26.065778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:26.065780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:26.065808 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:26.067284 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.067779 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.067782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.067783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.067888 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.068234 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.068237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.068238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.068516 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:26.068634 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.068953 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.069071 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 22:25:26.076440 25519 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:26.080057 25519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:26.099766 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.092625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.24.71.193:39383)
0811 22:25:26.094154 (+ 1529us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.094163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.094191 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:25:26.094283 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.094290 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.094527 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.094742 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:26.095049 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.096859 (+ 1810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.098220 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.098853 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.098993 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.099173 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.099458 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.099588 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1405,"thread_start_us":108,"threads_started":1}
I20260811 22:25:26.099817 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.093639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:34073 (local address 127.0.0.1:58472)
0811 22:25:26.093889 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:26.093905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:26.094014 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:26.094896 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:26.094899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:26.094920 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:26.095051 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.095059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.096395 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.096399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.098126 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.098134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.099295 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.099299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.099428 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.099468 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:25:26.099527 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 22:25:26.099634 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":171,"thread_start_us":113,"threads_started":1}
I20260811 22:25:26.100158 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.092518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33087 (local address 127.0.0.1:52092)
0811 22:25:26.093474 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:26.093545 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:26.093811 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:26.094940 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:26.094943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:26.094967 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:26.095188 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.095194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.096772 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.096775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.098123 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.098134 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.099728 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.099731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.099811 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.099868 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 22:25:26.099949 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:25:26.100026 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":155,"spinlock_wait_cycles":1408,"thread_start_us":107,"threads_started":1}
I20260811 22:25:26.100497 26049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.092382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:39383 (local address 127.0.0.1:37532)
0811 22:25:26.093625 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:26.093741 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:26.093843 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:26.094735 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:26.094741 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:26.094766 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:26.094926 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.094934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.097058 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.097061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.098058 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.098071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.099005 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.099009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.099128 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.099169 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:25:26.099303 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 22:25:26.100342 (+ 1039us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":873,"thread_start_us":115,"threads_started":1}
I20260811 22:25:26.100600 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.093251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.24.71.195:33087)
0811 22:25:26.094254 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.094263 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.094283 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:25:26.094399 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.094405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.094614 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.094766 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:26.095309 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.096582 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.098277 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.099400 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.099526 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.099666 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.100328 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.100446 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":181,"threads_started":1}
I20260811 22:25:26.100885 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.094107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.24.71.194:34073)
0811 22:25:26.094485 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.094495 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.094516 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:25:26.094584 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.094590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.094749 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.095066 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:26.095238 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.096211 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.098277 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.099144 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.100444 (+ 1300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.100598 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.100655 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.100759 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":162,"threads_started":1}
I20260811 22:25:26.104096 25810 tablet_service.cc:1467] Processing CreateTablet for tablet da1699e8d44c471884cad7f42611d20c (DEFAULT_TABLE table=test-table [id=8caf46611c7f415ba8c2a69394eaf98e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:26.105583 25810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da1699e8d44c471884cad7f42611d20c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:26.106206 25959 tablet_service.cc:1467] Processing CreateTablet for tablet da1699e8d44c471884cad7f42611d20c (DEFAULT_TABLE table=test-table [id=8caf46611c7f415ba8c2a69394eaf98e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:26.106431 25659 tablet_service.cc:1467] Processing CreateTablet for tablet da1699e8d44c471884cad7f42611d20c (DEFAULT_TABLE table=test-table [id=8caf46611c7f415ba8c2a69394eaf98e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:26.107328 25959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da1699e8d44c471884cad7f42611d20c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:26.107568 25659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da1699e8d44c471884cad7f42611d20c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:26.116721 26055 tablet_bootstrap.cc:492] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add: Bootstrap starting.
I20260811 22:25:26.117873 26058 tablet_bootstrap.cc:492] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766: Bootstrap starting.
I20260811 22:25:26.119154 26055 tablet_bootstrap.cc:654] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:26.120056 26055 log.cc:824] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:26.120175 26058 tablet_bootstrap.cc:654] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:26.120695 26056 tablet_bootstrap.cc:492] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3: Bootstrap starting.
I20260811 22:25:26.121803 26058 log.cc:824] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:26.122790 26055 tablet_bootstrap.cc:492] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add: No bootstrap required, opened a new log
I20260811 22:25:26.122983 26056 tablet_bootstrap.cc:654] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:26.123042 26055 ts_tablet_manager.cc:1397] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 22:25:26.124395 26056 log.cc:824] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:26.124590 26058 tablet_bootstrap.cc:492] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766: No bootstrap required, opened a new log
I20260811 22:25:26.124809 26058 ts_tablet_manager.cc:1397] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:25:26.126932 26056 tablet_bootstrap.cc:492] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3: No bootstrap required, opened a new log
I20260811 22:25:26.127161 26056 ts_tablet_manager.cc:1397] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260811 22:25:26.127467 26055 raft_consensus.cc:348] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } }
I20260811 22:25:26.127753 26055 raft_consensus.cc:374] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:26.127974 26055 raft_consensus.cc:729] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef6df919b4d49f18ba95f2f837a5add, State: Initialized, Role: FOLLOWER
I20260811 22:25:26.128801 26055 consensus_queue.cc:260] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } }
I20260811 22:25:26.129101 26058 raft_consensus.cc:348] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } }
I20260811 22:25:26.129455 26058 raft_consensus.cc:374] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:26.129519 26058 raft_consensus.cc:729] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b576a59c9efe4beab36263cb86825766, State: Initialized, Role: FOLLOWER
I20260811 22:25:26.129894 25736 heartbeater.cc:503] Master 127.24.71.254:41545 was elected leader, sending a full tablet report...
I20260811 22:25:26.130102 26055 ts_tablet_manager.cc:1428] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:25:26.130131 26058 consensus_queue.cc:260] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } }
I20260811 22:25:26.130884 25886 heartbeater.cc:503] Master 127.24.71.254:41545 was elected leader, sending a full tablet report...
I20260811 22:25:26.131357 26058 ts_tablet_manager.cc:1428] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:25:26.133524 26056 raft_consensus.cc:348] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } }
I20260811 22:25:26.133924 26056 raft_consensus.cc:374] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:26.134045 26056 raft_consensus.cc:729] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50bcb2985f6345939ea59304e54472e3, State: Initialized, Role: FOLLOWER
I20260811 22:25:26.134513 26056 consensus_queue.cc:260] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } }
I20260811 22:25:26.135632 26034 heartbeater.cc:503] Master 127.24.71.254:41545 was elected leader, sending a full tablet report...
I20260811 22:25:26.135890 26056 ts_tablet_manager.cc:1428] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260811 22:25:26.189625 26063 raft_consensus.cc:482] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:26.189910 26063 raft_consensus.cc:504] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } }
I20260811 22:25:26.191397 26063 leader_election.cc:288] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50bcb2985f6345939ea59304e54472e3 (127.24.71.195:33087), b576a59c9efe4beab36263cb86825766 (127.24.71.194:34073)
I20260811 22:25:26.197903 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.192795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:37807 (local address 127.24.71.194:34073)
0811 22:25:26.192947 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.192952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.193108 (+ 156us) server_negotiation.cc:407] Connection header received
0811 22:25:26.193314 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.193318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.193379 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.193486 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:26.193748 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.194490 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.196294 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.197002 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.197164 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.197226 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.197674 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.197770 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:25:26.197903 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.192022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:42181 (local address 127.24.71.195:33087)
0811 22:25:26.192196 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.192202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.192537 (+ 335us) server_negotiation.cc:407] Connection header received
0811 22:25:26.192727 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.192732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.192785 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.192842 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:26.193329 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.194202 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.196292 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.197024 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.197164 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.197226 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.197703 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.197800 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:25:26.197916 26068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.192566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:34073 (local address 127.24.71.193:37807)
0811 22:25:26.193024 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:26.193038 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:26.193123 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:26.193514 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:26.193516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:26.193535 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:26.193628 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.193632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.194635 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.194638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.196081 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.196095 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.197212 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.197217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.197438 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.197483 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 22:25:26.197585 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:25:26.197698 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":133,"thread_start_us":113,"threads_started":1}
I20260811 22:25:26.197917 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.192021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33087 (local address 127.24.71.193:42181)
0811 22:25:26.192412 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:26.192431 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:26.192567 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:26.193024 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:26.193027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:26.193051 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:26.193212 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.193223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.194448 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.194451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.196081 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:26.196095 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.197212 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.197217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:26.197428 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.197572 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 22:25:26.197660 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:25:26.197711 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":169,"thread_start_us":187,"threads_started":1}
I20260811 22:25:26.199024 25831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1699e8d44c471884cad7f42611d20c" candidate_uuid: "2ef6df919b4d49f18ba95f2f837a5add" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b576a59c9efe4beab36263cb86825766" is_pre_election: true
I20260811 22:25:26.199178 25980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1699e8d44c471884cad7f42611d20c" candidate_uuid: "2ef6df919b4d49f18ba95f2f837a5add" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50bcb2985f6345939ea59304e54472e3" is_pre_election: true
I20260811 22:25:26.199600 25831 raft_consensus.cc:2454] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ef6df919b4d49f18ba95f2f837a5add in term 0.
I20260811 22:25:26.200224 25608 leader_election.cc:302] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef6df919b4d49f18ba95f2f837a5add, b576a59c9efe4beab36263cb86825766; no voters:
I20260811 22:25:26.200536 26063 raft_consensus.cc:2781] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:26.200675 26063 raft_consensus.cc:482] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:26.200736 26063 raft_consensus.cc:3037] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:26.201589 25980 raft_consensus.cc:2454] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ef6df919b4d49f18ba95f2f837a5add in term 0.
I20260811 22:25:26.202162 26063 raft_consensus.cc:504] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } }
I20260811 22:25:26.202677 26063 leader_election.cc:288] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [CANDIDATE]: Term 1 election: Requested vote from peers 50bcb2985f6345939ea59304e54472e3 (127.24.71.195:33087), b576a59c9efe4beab36263cb86825766 (127.24.71.194:34073)
I20260811 22:25:26.203321 25831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1699e8d44c471884cad7f42611d20c" candidate_uuid: "2ef6df919b4d49f18ba95f2f837a5add" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b576a59c9efe4beab36263cb86825766"
I20260811 22:25:26.203292 25980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da1699e8d44c471884cad7f42611d20c" candidate_uuid: "2ef6df919b4d49f18ba95f2f837a5add" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50bcb2985f6345939ea59304e54472e3"
I20260811 22:25:26.203470 25831 raft_consensus.cc:3037] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:26.203562 25980 raft_consensus.cc:3037] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:26.205372 25980 raft_consensus.cc:2454] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef6df919b4d49f18ba95f2f837a5add in term 1.
I20260811 22:25:26.205372 25831 raft_consensus.cc:2454] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ef6df919b4d49f18ba95f2f837a5add in term 1.
I20260811 22:25:26.205922 25608 leader_election.cc:302] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ef6df919b4d49f18ba95f2f837a5add, b576a59c9efe4beab36263cb86825766; no voters:
I20260811 22:25:26.206194 26063 raft_consensus.cc:2781] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:26.206483 26063 raft_consensus.cc:686] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [term 1 LEADER]: Becoming Leader. State: Replica: 2ef6df919b4d49f18ba95f2f837a5add, State: Running, Role: LEADER
I20260811 22:25:26.206930 26063 consensus_queue.cc:237] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } }
I20260811 22:25:26.210774 25519 catalog_manager.cc:5495] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ef6df919b4d49f18ba95f2f837a5add (127.24.71.193). New cstate: current_term: 1 leader_uuid: "2ef6df919b4d49f18ba95f2f837a5add" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef6df919b4d49f18ba95f2f837a5add" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50bcb2985f6345939ea59304e54472e3" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b576a59c9efe4beab36263cb86825766" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34073 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:25:26 dist-test-slave-mvvj krb5kdc[25465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:25:26.230568 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.219159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.24.71.193:39383)
0811 22:25:26.219376 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.219384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.219804 (+ 420us) server_negotiation.cc:407] Connection header received
0811 22:25:26.219946 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.219948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.219997 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.220076 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:26.220914 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.221707 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.222688 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.222936 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.223041 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.226244 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:26.226259 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:26.226276 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:26.226307 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:26.228332 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.228894 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.228900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.228902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.229011 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.229351 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.229354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.229355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.229702 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:26.229977 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.230220 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.230367 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
Aug 11 22:25:26 dist-test-slave-mvvj krb5kdc[25465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487126, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:25:26 dist-test-slave-mvvj krb5kdc[25465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487126, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
W20260811 22:25:26.262249 25737 tablet.cc:2369] T da1699e8d44c471884cad7f42611d20c P 2ef6df919b4d49f18ba95f2f837a5add: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:26.264122 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.253400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.24.71.193:39383)
0811 22:25:26.253577 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.253582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.253817 (+ 235us) server_negotiation.cc:407] Connection header received
0811 22:25:26.253984 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.253987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.254036 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.254113 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:26.254953 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.255503 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.256480 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.256779 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.256909 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.259950 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:26.259955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:26.259957 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:26.259993 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:26.262107 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.262731 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.262734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.262736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.262828 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.263145 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.263148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.263149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.263353 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:26.263472 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.263813 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.263991 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 22:25:26.274737 26037 tablet.cc:2369] T da1699e8d44c471884cad7f42611d20c P 50bcb2985f6345939ea59304e54472e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:25:26.277436 25887 tablet.cc:2369] T da1699e8d44c471884cad7f42611d20c P b576a59c9efe4beab36263cb86825766: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:25:26 dist-test-slave-mvvj krb5kdc[25465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:25:26 dist-test-slave-mvvj krb5kdc[25465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:25:26.295768 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:26.285279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.24.71.193:39383)
0811 22:25:26.285571 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:26.285575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:26.285591 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:26.285800 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:26.285803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:26.285855 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:26.285960 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:26.286812 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.287367 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.288421 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:26.288657 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:26.288757 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:26.291695 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:26.291699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:26.291701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:26.291741 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:26.293483 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.294066 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.294070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.294071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.294160 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:26.294658 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:26.294661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:26.294662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:26.294934 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:26.295076 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:26.295383 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 22:25:26.295515 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
W20260811 22:25:26.297888 25637 tablet_service.cc:2908] Rejecting scan request for tablet da1699e8d44c471884cad7f42611d20c: Uninitialized: clean time has not yet been initialized
I20260811 22:25:26.301288 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 25588
I20260811 22:25:26.311990 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 25741
I20260811 22:25:26.322290 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 25891
I20260811 22:25:26.332034 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 25480
2026-08-11T22:25:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2731 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:26 dist-test-slave-mvvj krb5kdc[26085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:26 dist-test-slave-mvvj krb5kdc[26085](info): set up 2 sockets
Aug 11 22:25:26 dist-test-slave-mvvj krb5kdc[26085](info): commencing operation
krb5kdc: starting...
W20260811 22:25:27.487632 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:27 dist-test-slave-mvvj krb5kdc[26085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:27.644316 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38211
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:38211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:27.771716 26101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:27.772100 26101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:27.772197 26101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:27.776556 26101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:27.776657 26101 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:25:27.776690 26101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:27.776737 26101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:27.776768 26101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:27.776809 26101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:27.783423 26101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:38211
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:27.784708 26101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:27.785771 26101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:27.796535 26106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:27.797008 26101 server_base.cc:1034] running on GCE node
W20260811 22:25:27.797013 26109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:27.797046 26107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:27.797981 26101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:27.799787 26101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:27.801061 26101 hybrid_clock.cc:648] HybridClock initialized: now 1786487127801021 us; error 53 us; skew 500 ppm
Aug 11 22:25:27 dist-test-slave-mvvj krb5kdc[26085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:27.804437 26101 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:27.805440 26101 webserver.cc:460] Webserver started at http://127.24.71.254:35765/ using document root <none> and password file <none>
I20260811 22:25:27.806015 26101 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:27.806079 26101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:27.806262 26101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:27.808529 26101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "5fd1bc56364e48d49c34c28cf817b5f9"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-mvvj"
server_key: "ebc4951f87fdf0b6482966389fa6f211"
server_key_iv: "9e7cb9e7ffde11ac431ad60d837beb0a"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.809100 26101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "5fd1bc56364e48d49c34c28cf817b5f9"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-mvvj"
server_key: "ebc4951f87fdf0b6482966389fa6f211"
server_key_iv: "9e7cb9e7ffde11ac431ad60d837beb0a"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.813314 26101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:27.816284 26118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:27.817848 26101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:25:27.818032 26101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "5fd1bc56364e48d49c34c28cf817b5f9"
format_stamp: "Formatted at 2026-08-11 22:25:27 on dist-test-slave-mvvj"
server_key: "ebc4951f87fdf0b6482966389fa6f211"
server_key_iv: "9e7cb9e7ffde11ac431ad60d837beb0a"
server_key_version: "encryptionkey@0"
I20260811 22:25:27.818149 26101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:27.832233 26101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:27.835601 26101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:27.835824 26101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:27.844947 26101 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:38211
I20260811 22:25:27.844976 26188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:38211 every 8 connection(s)
I20260811 22:25:27.845971 26101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:27.849012 26189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:27.853214 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 26101
I20260811 22:25:27.853348 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:27.853659 24863 external_mini_cluster.cc:1428] Setting key c1eebf35add7da9c62034c12b58cd83b
I20260811 22:25:27.855759 26189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9: Bootstrap starting.
I20260811 22:25:27.858283 26189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:27.859161 26189 log.cc:824] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:27.861446 26189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9: No bootstrap required, opened a new log
Aug 11 22:25:27 dist-test-slave-mvvj krb5kdc[26085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:27.866014 26189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd1bc56364e48d49c34c28cf817b5f9" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38211 } }
I20260811 22:25:27.866276 26189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:27.866310 26189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd1bc56364e48d49c34c28cf817b5f9, State: Initialized, Role: FOLLOWER
I20260811 22:25:27.866840 26189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd1bc56364e48d49c34c28cf817b5f9" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38211 } }
I20260811 22:25:27.866973 26189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:27.867069 26189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:27.867213 26189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:27.868623 26189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd1bc56364e48d49c34c28cf817b5f9" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38211 } }
I20260811 22:25:27.869009 26189 leader_election.cc:302] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fd1bc56364e48d49c34c28cf817b5f9; no voters:
I20260811 22:25:27.869390 26189 leader_election.cc:288] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:27.869585 26194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:27.869824 26194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [term 1 LEADER]: Becoming Leader. State: Replica: 5fd1bc56364e48d49c34c28cf817b5f9, State: Running, Role: LEADER
I20260811 22:25:27.869900 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:27.855348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.24.71.254:38211)
0811 22:25:27.855947 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:27.855986 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:27.856026 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:25:27.856798 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:27.856819 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:27.857197 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:27.857577 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:27.858503 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.859634 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.860658 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:27.861232 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:27.861424 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:27.864361 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:27.864375 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:27.864391 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:27.864424 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:27.866334 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.866856 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.866862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.866867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.867062 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:27.867390 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:27.867393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:27.867395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:27.867968 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:27.868176 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:27.868476 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 22:25:27.868815 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":152,"threads_started":1}
I20260811 22:25:27.870180 26194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd1bc56364e48d49c34c28cf817b5f9" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38211 } }
I20260811 22:25:27.870980 26189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:27.872102 26195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fd1bc56364e48d49c34c28cf817b5f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd1bc56364e48d49c34c28cf817b5f9" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38211 } } }
I20260811 22:25:27.872242 26195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:27.872087 26196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fd1bc56364e48d49c34c28cf817b5f9. Latest consensus state: current_term: 1 leader_uuid: "5fd1bc56364e48d49c34c28cf817b5f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd1bc56364e48d49c34c28cf817b5f9" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38211 } } }
I20260811 22:25:27.872351 26196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:27.872649 26206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:27.875655 26206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:27.882211 26206 catalog_manager.cc:1347] Generated new cluster ID: ab885aa83d5e4bd6b1a1e8a2a3835307
I20260811 22:25:27.882308 26206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:27.911626 26206 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:27.912767 26206 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:27.917758 26206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9: Generated new TSK 0
I20260811 22:25:27.918646 26206 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:25:27.959287 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38211
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:28.081275 26219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:28.081579 26219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:28.081627 26219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:28.085852 26219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:28.085932 26219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:28.085973 26219 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:25:28.086054 26219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:28.092100 26219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.71.254:38211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:28.093351 26219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:28.094467 26219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:28.105384 26226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:28.105640 26219 server_base.cc:1034] running on GCE node
W20260811 22:25:28.105465 26228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.105515 26225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:28.106268 26219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:28.107039 26219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:28.108229 26219 hybrid_clock.cc:648] HybridClock initialized: now 1786487128108198 us; error 43 us; skew 500 ppm
Aug 11 22:25:28 dist-test-slave-mvvj krb5kdc[26085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487128, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:28.111129 26219 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:25:28.112219 26219 webserver.cc:460] Webserver started at http://127.24.71.193:46247/ using document root <none> and password file <none>
I20260811 22:25:28.112843 26219 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:28.112941 26219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:28.113190 26219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:28.115665 26219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "bfa42b46c70846c6a2c7d15401344a66"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-mvvj"
server_key: "25b9e454491a0314a28a6e3f17ed7ea6"
server_key_iv: "12a7e2f751565ab57a11f201268b7548"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.116438 26219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "bfa42b46c70846c6a2c7d15401344a66"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-mvvj"
server_key: "25b9e454491a0314a28a6e3f17ed7ea6"
server_key_iv: "12a7e2f751565ab57a11f201268b7548"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.120895 26219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:28.123769 26235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.125262 26219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:28.125502 26219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "bfa42b46c70846c6a2c7d15401344a66"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-mvvj"
server_key: "25b9e454491a0314a28a6e3f17ed7ea6"
server_key_iv: "12a7e2f751565ab57a11f201268b7548"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.125772 26219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:28.149966 26219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:28.153611 26219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:28.153846 26219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:28.155562 26219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:28.155645 26219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.155725 26219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:28.155769 26219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:25:28 dist-test-slave-mvvj krb5kdc[26085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487128, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:28.167528 26219 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:45835
I20260811 22:25:28.167572 26365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:45835 every 8 connection(s)
I20260811 22:25:28.168612 26219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:28.171290 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.156408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:55759 (local address 127.24.71.254:38211)
0811 22:25:28.156585 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.156592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.158189 (+ 1597us) server_negotiation.cc:407] Connection header received
0811 22:25:28.159398 (+ 1209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.159403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.159465 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.159582 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.161575 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.162539 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.164122 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.164529 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.164684 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.167484 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.167489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.167492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:28.167530 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.169314 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.169842 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.169845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.169846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.169937 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.170258 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.170260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.170261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.170452 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.170536 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.171027 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.171159 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:25:28.172192 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.157464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38211 (local address 127.24.71.193:55759)
0811 22:25:28.158023 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.158078 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.159156 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.159843 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.159900 (+ 57us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.160535 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.161358 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.161385 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.162692 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.162697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.163970 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.163981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.164713 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.164716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.164867 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.165596 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:28.165625 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.167251 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.169463 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.169469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.169486 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.169754 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.170034 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.170036 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.170037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.170170 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.170546 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.170552 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.170784 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.170941 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.171208 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":129,"threads_started":1}
I20260811 22:25:28.173590 26366 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38211
I20260811 22:25:28.173843 26366 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:28.174432 26366 heartbeater.cc:511] Master 127.24.71.254:38211 requested a full tablet report, sending...
I20260811 22:25:28.176411 26137 ts_manager.cc:194] Registered new tserver with Master: bfa42b46c70846c6a2c7d15401344a66 (127.24.71.193:45835)
I20260811 22:25:28.177716 26137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:55759
I20260811 22:25:28.178143 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 26219
I20260811 22:25:28.178264 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:28.178613 24863 external_mini_cluster.cc:1428] Setting key 0f93ce7e6330293e88a044153dc7548c
I20260811 22:25:28.185812 26372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.176229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38211 (local address 127.24.71.193:59161)
0811 22:25:28.176584 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.176600 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.176704 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.178807 (+ 2103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.178810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.179084 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.179247 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.179256 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.180154 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.180158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.180746 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.180755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.181257 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.181260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.181355 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.181930 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:28.181942 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.182292 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.184163 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.184168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.184170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.184387 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.184679 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.184683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.184685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.184817 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.185232 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.185237 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.185337 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.185447 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.185620 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":111,"threads_started":1}
I20260811 22:25:28.185802 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.176375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:59161 (local address 127.24.71.254:38211)
0811 22:25:28.178566 (+ 2191us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.178571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.178586 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:25:28.178629 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.178634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.178686 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.178778 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.179381 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.180027 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.180886 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.181130 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.181248 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.182442 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.182446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.182448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:28.182486 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.184048 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.184494 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.184498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.184501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.184581 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.184922 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.184925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.184927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.185121 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.185229 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.185512 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.185636 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2104}
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:25:28.212718 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38211
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:28.333734 26375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:28.334039 26375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:28.334088 26375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:28.338416 26375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:28.338596 26375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:28.338649 26375 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:25:28.338685 26375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:28.344540 26375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.71.254:38211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:28.345678 26375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:28.346728 26375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:28.357630 26384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.357626 26380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.357626 26381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:28.358001 26375 server_base.cc:1034] running on GCE node
I20260811 22:25:28.358587 26375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:28.359403 26375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:28.360605 26375 hybrid_clock.cc:648] HybridClock initialized: now 1786487128360542 us; error 76 us; skew 500 ppm
Aug 11 22:25:28 dist-test-slave-mvvj krb5kdc[26085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487128, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:28.363945 26375 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:25:28.364986 26375 webserver.cc:460] Webserver started at http://127.24.71.194:44187/ using document root <none> and password file <none>
I20260811 22:25:28.365659 26375 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:28.365731 26375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:28.365976 26375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:28.368327 26375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "9d59a980446d4750b59c93b02a859d0a"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-mvvj"
server_key: "ab83c48ad01a15b0925d7c1b76c69bd8"
server_key_iv: "cf2295000f4e4f94cce2e6dab023bded"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.368956 26375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "9d59a980446d4750b59c93b02a859d0a"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-mvvj"
server_key: "ab83c48ad01a15b0925d7c1b76c69bd8"
server_key_iv: "cf2295000f4e4f94cce2e6dab023bded"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.373512 26375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:28.376422 26392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.377887 26375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:28.378044 26375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "9d59a980446d4750b59c93b02a859d0a"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-mvvj"
server_key: "ab83c48ad01a15b0925d7c1b76c69bd8"
server_key_iv: "cf2295000f4e4f94cce2e6dab023bded"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.378173 26375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:28.390627 26375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:28.394416 26375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:28.394649 26375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:28.396405 26375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:28.396471 26375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.396538 26375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:28.396570 26375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.408320 26375 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:36571
I20260811 22:25:28.408393 26522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:36571 every 8 connection(s)
Aug 11 22:25:28 dist-test-slave-mvvj krb5kdc[26085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487128, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:28.409487 26375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:28.411161 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 26375
I20260811 22:25:28.411288 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:28.411635 24863 external_mini_cluster.cc:1428] Setting key 81a9eea0fa303f9ab87756315cecb1f2
I20260811 22:25:28.414218 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.397372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:45945 (local address 127.24.71.254:38211)
0811 22:25:28.397523 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.397528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.398733 (+ 1205us) server_negotiation.cc:407] Connection header received
0811 22:25:28.400172 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.400177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.400254 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.400369 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.402392 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.403331 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.405345 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.405723 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.405868 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.409309 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.409313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.409316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:28.409355 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.411546 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.412299 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.412307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.412310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.412424 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.412864 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.412868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.412870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.413128 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.413261 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.413907 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.414052 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:25:28.415187 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.397966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38211 (local address 127.24.71.194:45945)
0811 22:25:28.398547 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.398598 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.399786 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.400639 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.400650 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.401269 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.402161 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.402191 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.403500 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.403503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.405165 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.405178 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.405929 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.405937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.406220 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.407121 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:28.407145 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.409080 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.411754 (+ 2674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.411762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.411781 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.412157 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.412541 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.412547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.412549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.412734 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.413283 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.413291 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.413611 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.413825 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.414298 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":158,"threads_started":1}
I20260811 22:25:28.416870 26523 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38211
I20260811 22:25:28.417059 26523 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:28.417770 26523 heartbeater.cc:511] Master 127.24.71.254:38211 requested a full tablet report, sending...
I20260811 22:25:28.419811 26135 ts_manager.cc:194] Registered new tserver with Master: 9d59a980446d4750b59c93b02a859d0a (127.24.71.194:36571)
I20260811 22:25:28.420657 26135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:45945
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:25:28.429719 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.420061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38211 (local address 127.24.71.194:35097)
0811 22:25:28.420399 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.420415 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.420524 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.420962 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.420966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.421264 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.421448 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.421474 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.422847 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.422852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.423960 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.423976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.424533 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.424541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.424689 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.425267 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:28.425286 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.425789 (+ 503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.427697 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.427704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.427710 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.428041 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.428332 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.428334 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.428336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.428471 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.429004 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.429011 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.429144 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.429234 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.429370 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
I20260811 22:25:28.429819 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.419739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:35097 (local address 127.24.71.254:38211)
0811 22:25:28.419936 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.419943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.420513 (+ 570us) server_negotiation.cc:407] Connection header received
0811 22:25:28.420625 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.420632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.420702 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.420794 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.421643 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.422653 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.424111 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.424370 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.424477 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.425967 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.425971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.425973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:28.426004 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.427552 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.428149 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.428153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.428154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.428242 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.428599 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.428602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.428603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.428895 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.428984 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.429352 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.429514 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:25:28.447983 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38211
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:28.572067 26533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:28.572417 26533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:28.572504 26533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:28.577039 26533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:28.577142 26533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:28.577212 26533 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:25:28.577294 26533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:28.583160 26533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.71.254:38211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:28.584498 26533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:28.585611 26533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:28.596210 26541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:28.596210 26538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:28.596251 26533 server_base.cc:1034] running on GCE node
W20260811 22:25:28.596210 26539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:28.596880 26533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:28.597615 26533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:28.598820 26533 hybrid_clock.cc:648] HybridClock initialized: now 1786487128598775 us; error 61 us; skew 500 ppm
Aug 11 22:25:28 dist-test-slave-mvvj krb5kdc[26085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487128, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:28.601897 26533 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:25:28.602900 26533 webserver.cc:460] Webserver started at http://127.24.71.195:36313/ using document root <none> and password file <none>
I20260811 22:25:28.603456 26533 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:28.603518 26533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:28.603693 26533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:28.606312 26533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "80b8b52ea83b4a628087e33c1a171932"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-mvvj"
server_key: "081786c22b2073260b9483316e2c3a58"
server_key_iv: "41a8a1b2d809e1f47c9665db1dc59128"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.606889 26533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "80b8b52ea83b4a628087e33c1a171932"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-mvvj"
server_key: "081786c22b2073260b9483316e2c3a58"
server_key_iv: "41a8a1b2d809e1f47c9665db1dc59128"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.610889 26533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:25:28.613581 26548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.615043 26533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:28.615191 26533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "80b8b52ea83b4a628087e33c1a171932"
format_stamp: "Formatted at 2026-08-11 22:25:28 on dist-test-slave-mvvj"
server_key: "081786c22b2073260b9483316e2c3a58"
server_key_iv: "41a8a1b2d809e1f47c9665db1dc59128"
server_key_version: "encryptionkey@0"
I20260811 22:25:28.615320 26533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:28.630755 26533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:28.634033 26533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:28.634271 26533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:28.635989 26533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:28.636073 26533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:28.636147 26533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:28.636178 26533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:25:28 dist-test-slave-mvvj krb5kdc[26085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487128, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:28.647949 26533 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:38441
I20260811 22:25:28.648373 26684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:38441 every 8 connection(s)
I20260811 22:25:28.648962 26533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:28.651444 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.637022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:39053 (local address 127.24.71.254:38211)
0811 22:25:28.637223 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.637230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.638106 (+ 876us) server_negotiation.cc:407] Connection header received
0811 22:25:28.639214 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.639221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.639317 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.639382 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.641146 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.641818 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.643147 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.643435 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.643578 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.647369 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.647372 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.647375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:28.647403 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.649095 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.649761 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.649767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.649771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.649870 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.650257 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.650261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.650263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.650474 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.650589 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.651122 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.651255 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:25:28.652284 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.637434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38211 (local address 127.24.71.195:39053)
0811 22:25:28.637920 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.637976 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.638917 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.639937 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.639945 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.640393 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.640991 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.641006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.641952 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.641956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.643013 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.643024 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.643641 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.643648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.643935 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.644737 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:28.644757 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.647155 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.649225 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.649230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.649246 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.649617 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.649999 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.650003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.650005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.650152 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.650598 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.650605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.650859 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.651050 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.651389 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":139,"threads_started":1}
I20260811 22:25:28.653586 26685 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38211
I20260811 22:25:28.653765 26685 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:28.654274 26685 heartbeater.cc:511] Master 127.24.71.254:38211 requested a full tablet report, sending...
I20260811 22:25:28.655503 26135 ts_manager.cc:194] Registered new tserver with Master: 80b8b52ea83b4a628087e33c1a171932 (127.24.71.195:38441)
I20260811 22:25:28.656344 26135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:39053
I20260811 22:25:28.656670 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 26533
I20260811 22:25:28.656822 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:28.657179 24863 external_mini_cluster.cc:1428] Setting key 223dace8010a590c21bea91b44061072
I20260811 22:25:28.659621 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:28.664698 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.655993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:36387 (local address 127.24.71.254:38211)
0811 22:25:28.656133 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.656138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.656408 (+ 270us) server_negotiation.cc:407] Connection header received
0811 22:25:28.656584 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.656588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.656646 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.656740 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.657428 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.658163 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.659293 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.659632 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.659760 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.661205 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.661208 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.661210 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:28.661234 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.662752 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.663298 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.663300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.663302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.663384 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.663713 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.663715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.663716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.663993 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.664076 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.664453 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.664572 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:25:28.664834 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.655882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38211 (local address 127.24.71.195:36387)
0811 22:25:28.656297 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.656312 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.656442 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.656880 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.656886 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.657136 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.657293 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.657302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.658339 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.658343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.659151 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.659177 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.659776 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.659780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.659891 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.660458 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:28.660519 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.661040 (+ 521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.662874 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.662880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.662884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.663180 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.663494 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.663500 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.663505 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.663613 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.664094 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.664100 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.664225 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.664349 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.664536 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":2,"thread_start_us":120,"threads_started":1}
Aug 11 22:25:28 dist-test-slave-mvvj krb5kdc[26085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487128, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:25:28 dist-test-slave-mvvj krb5kdc[26085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487128, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:28.692581 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.681808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.24.71.254:38211)
0811 22:25:28.681978 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.681982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.681998 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:28.682049 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.682051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.682096 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.682178 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.683315 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.684082 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.685120 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.685352 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.685452 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.688717 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.688721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.688723 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:28.688764 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.690380 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.690907 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.690910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.690912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.691004 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.691411 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.691414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.691415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.691635 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.691746 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.692162 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.692292 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:25:28.696473 26135 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:28.700002 26135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:28.717423 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.710078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:36571 (local address 127.0.0.1:44902)
0811 22:25:28.710775 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.710789 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.710910 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.711762 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.711768 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.711791 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:28.711989 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.711999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.713779 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.713783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.715566 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.715580 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.716998 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.717002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.717086 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.717117 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.717182 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.717272 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":186,"spinlock_wait_cycles":1792,"thread_start_us":108,"threads_started":1}
I20260811 22:25:28.717468 26707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.710758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.24.71.194:36571)
0811 22:25:28.711252 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.711270 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.711336 (+ 66us) server_negotiation.cc:407] Connection header received
0811 22:25:28.711412 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.711416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.711589 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.711758 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:28.712136 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.713411 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.715970 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.716832 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.716955 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.717144 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.717216 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.717314 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":116,"threads_started":1}
I20260811 22:25:28.718017 26703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.709749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:45835 (local address 127.0.0.1:34562)
0811 22:25:28.710127 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.710188 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.710428 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.711968 (+ 1540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.711971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.711996 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:28.712181 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.712190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.713498 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.713505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.715566 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.715580 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.716637 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.716641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.717631 (+ 990us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.717695 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.717784 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.717865 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":1096,"thread_start_us":109,"threads_started":1}
I20260811 22:25:28.718155 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.710975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37704 (local address 127.24.71.195:38441)
0811 22:25:28.711501 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.711515 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.711538 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:25:28.711618 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.711622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.711816 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.712033 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:28.712311 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.713421 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.715576 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.716458 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.716598 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.716835 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.717905 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.718006 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":142,"threads_started":1}
I20260811 22:25:28.717391 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.710522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:38441 (local address 127.0.0.1:37704)
0811 22:25:28.710803 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.710819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.710910 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.712013 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.712016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.712029 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:28.712175 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.712190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.713590 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.713594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.715418 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.715433 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.716645 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.716650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.716815 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.716877 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.717017 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.717186 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":225,"spinlock_wait_cycles":9216,"thread_start_us":100,"threads_started":1}
I20260811 22:25:28.718897 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.709950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.24.71.193:45835)
0811 22:25:28.711187 (+ 1237us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.711195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.711221 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:25:28.711348 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.711355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.711590 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.711758 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:28.712352 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.713349 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.715719 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.716480 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.716701 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.716845 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.718658 (+ 1813us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.718770 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1114,"thread_start_us":111,"threads_started":1}
I20260811 22:25:28.721647 26608 tablet_service.cc:1467] Processing CreateTablet for tablet 886e8027a337401a8a2bf0aa1fb627c3 (DEFAULT_TABLE table=test-table [id=a24d596cbb454aedad9d27d09981c46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:28.721640 26292 tablet_service.cc:1467] Processing CreateTablet for tablet 886e8027a337401a8a2bf0aa1fb627c3 (DEFAULT_TABLE table=test-table [id=a24d596cbb454aedad9d27d09981c46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:28.722784 26292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 886e8027a337401a8a2bf0aa1fb627c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.722797 26608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 886e8027a337401a8a2bf0aa1fb627c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.727922 26449 tablet_service.cc:1467] Processing CreateTablet for tablet 886e8027a337401a8a2bf0aa1fb627c3 (DEFAULT_TABLE table=test-table [id=a24d596cbb454aedad9d27d09981c46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:28.729218 26449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 886e8027a337401a8a2bf0aa1fb627c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.729386 26709 tablet_bootstrap.cc:492] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: Bootstrap starting.
I20260811 22:25:28.731244 26710 tablet_bootstrap.cc:492] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: Bootstrap starting.
I20260811 22:25:28.732033 26709 tablet_bootstrap.cc:654] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.733286 26709 log.cc:824] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:28.733680 26710 tablet_bootstrap.cc:654] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.734611 26710 log.cc:824] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:28.735466 26709 tablet_bootstrap.cc:492] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: No bootstrap required, opened a new log
I20260811 22:25:28.735723 26709 ts_tablet_manager.cc:1397] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260811 22:25:28.737053 26710 tablet_bootstrap.cc:492] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: No bootstrap required, opened a new log
I20260811 22:25:28.737267 26710 ts_tablet_manager.cc:1397] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:28.738512 26713 tablet_bootstrap.cc:492] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: Bootstrap starting.
I20260811 22:25:28.741194 26710 raft_consensus.cc:348] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } }
I20260811 22:25:28.741443 26710 raft_consensus.cc:374] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.741237 26709 raft_consensus.cc:348] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } }
I20260811 22:25:28.741528 26710 raft_consensus.cc:729] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b8b52ea83b4a628087e33c1a171932, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.741549 26709 raft_consensus.cc:374] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.741638 26709 raft_consensus.cc:729] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa42b46c70846c6a2c7d15401344a66, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.742035 26710 consensus_queue.cc:260] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } }
I20260811 22:25:28.742122 26709 consensus_queue.cc:260] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } }
I20260811 22:25:28.742398 26713 tablet_bootstrap.cc:654] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.742820 26685 heartbeater.cc:503] Master 127.24.71.254:38211 was elected leader, sending a full tablet report...
I20260811 22:25:28.743185 26710 ts_tablet_manager.cc:1428] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:25:28.743254 26709 ts_tablet_manager.cc:1428] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:25:28.743625 26366 heartbeater.cc:503] Master 127.24.71.254:38211 was elected leader, sending a full tablet report...
I20260811 22:25:28.743649 26714 raft_consensus.cc:482] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:28.743914 26714 raft_consensus.cc:504] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } }
I20260811 22:25:28.743448 26713 log.cc:824] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:28.745306 26714 leader_election.cc:288] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa42b46c70846c6a2c7d15401344a66 (127.24.71.193:45835), 9d59a980446d4750b59c93b02a859d0a (127.24.71.194:36571)
I20260811 22:25:28.747000 26713 tablet_bootstrap.cc:492] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: No bootstrap required, opened a new log
I20260811 22:25:28.747218 26713 ts_tablet_manager.cc:1397] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 22:25:28.750537 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.746130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:42077 (local address 127.24.71.193:45835)
0811 22:25:28.746253 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.746258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.746273 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:25:28.746337 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.746340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.746385 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.746477 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:28.746888 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.747624 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.749211 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.749905 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.750061 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.750117 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.750342 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.750418 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:25:28.750537 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.745480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:45835 (local address 127.24.71.195:42077)
0811 22:25:28.745723 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.745748 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.745862 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.746607 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.746612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.746632 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:28.746747 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.746754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.747734 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.747738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.749051 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.749063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.750052 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.750057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.750172 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.750198 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.750306 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.750391 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 22:25:28.751152 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "886e8027a337401a8a2bf0aa1fb627c3" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa42b46c70846c6a2c7d15401344a66" is_pre_election: true
I20260811 22:25:28.751483 26315 raft_consensus.cc:2454] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 0.
I20260811 22:25:28.752045 26550 leader_election.cc:302] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80b8b52ea83b4a628087e33c1a171932, bfa42b46c70846c6a2c7d15401344a66; no voters:
I20260811 22:25:28.752358 26714 raft_consensus.cc:2781] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:28.752463 26714 raft_consensus.cc:482] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:28.752511 26714 raft_consensus.cc:3037] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:28.753242 26707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.746748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:51233 (local address 127.24.71.194:36571)
0811 22:25:28.746875 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.746879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.748148 (+ 1269us) server_negotiation.cc:407] Connection header received
0811 22:25:28.748343 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.748346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.748403 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.748453 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:28.749557 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.750236 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.752056 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.752604 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.752770 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.752827 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.753042 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.753118 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 22:25:28.753268 26718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.746297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:36571 (local address 127.24.71.195:51233)
0811 22:25:28.748058 (+ 1761us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.748075 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.748219 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.748596 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.748599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.748617 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:28.749410 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.749423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.750380 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.750383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.751911 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.751925 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.752770 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.752775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.752893 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.752925 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.753025 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.753102 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1694,"spinlock_wait_cycles":2432,"thread_start_us":147,"threads_started":1}
I20260811 22:25:28.753890 26714 raft_consensus.cc:504] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } }
I20260811 22:25:28.753881 26471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "886e8027a337401a8a2bf0aa1fb627c3" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d59a980446d4750b59c93b02a859d0a" is_pre_election: true
I20260811 22:25:28.753964 26713 raft_consensus.cc:348] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } }
I20260811 22:25:28.754189 26713 raft_consensus.cc:374] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.754245 26713 raft_consensus.cc:729] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d59a980446d4750b59c93b02a859d0a, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.754779 26714 leader_election.cc:288] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 election: Requested vote from peers bfa42b46c70846c6a2c7d15401344a66 (127.24.71.193:45835), 9d59a980446d4750b59c93b02a859d0a (127.24.71.194:36571)
I20260811 22:25:28.754766 26469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "886e8027a337401a8a2bf0aa1fb627c3" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d59a980446d4750b59c93b02a859d0a"
I20260811 22:25:28.754788 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "886e8027a337401a8a2bf0aa1fb627c3" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa42b46c70846c6a2c7d15401344a66"
I20260811 22:25:28.754806 26713 consensus_queue.cc:260] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } }
I20260811 22:25:28.754902 26315 raft_consensus.cc:3037] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:28.755560 26523 heartbeater.cc:503] Master 127.24.71.254:38211 was elected leader, sending a full tablet report...
I20260811 22:25:28.755795 26471 raft_consensus.cc:2454] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 0.
I20260811 22:25:28.756057 26713 ts_tablet_manager.cc:1428] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 22:25:28.756491 26315 raft_consensus.cc:2454] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 1.
I20260811 22:25:28.756801 26550 leader_election.cc:302] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80b8b52ea83b4a628087e33c1a171932, bfa42b46c70846c6a2c7d15401344a66; no voters: 9d59a980446d4750b59c93b02a859d0a
I20260811 22:25:28.756999 26714 raft_consensus.cc:2781] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:28.757117 26714 raft_consensus.cc:686] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 1 LEADER]: Becoming Leader. State: Replica: 80b8b52ea83b4a628087e33c1a171932, State: Running, Role: LEADER
I20260811 22:25:28.757541 26714 consensus_queue.cc:237] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } }
I20260811 22:25:28.761075 26137 catalog_manager.cc:5495] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80b8b52ea83b4a628087e33c1a171932 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } health_report { overall_health: UNKNOWN } } }
I20260811 22:25:28.791481 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.787368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.24.71.195:38441)
0811 22:25:28.787536 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.787544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.787562 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:25:28.787883 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.787886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.787958 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.788085 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:28.788374 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.788906 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.789819 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.790055 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.790167 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.790561 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:28.791014 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:28.791149 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.791259 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.791327 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:25:28.798666 26315 raft_consensus.cc:1261] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Refusing update from remote peer 80b8b52ea83b4a628087e33c1a171932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:28.799734 26471 raft_consensus.cc:3037] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:28.799724 26714 consensus_queue.cc:1035] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:28.801961 26471 raft_consensus.cc:1261] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a [term 1 FOLLOWER]: Refusing update from remote peer 80b8b52ea83b4a628087e33c1a171932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:28.802627 26714 consensus_queue.cc:1035] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:28.808594 26722 mvcc.cc:205] Tried to move back new op lower bound from 7317451279549779968 to 7317451279394304000. Current Snapshot: MvccSnapshot[applied={T|T < 7317451279549779968}]
I20260811 22:25:28.812327 26726 mvcc.cc:205] Tried to move back new op lower bound from 7317451279549779968 to 7317451279394304000. Current Snapshot: MvccSnapshot[applied={T|T < 7317451279549779968}]
I20260811 22:25:28.823493 26137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:28.823773 26137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:28.826807 26137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9: Sending DeleteTablet for 3 replicas of tablet 886e8027a337401a8a2bf0aa1fb627c3
I20260811 22:25:28.828033 26292 tablet_service.cc:1514] Processing DeleteTablet for tablet 886e8027a337401a8a2bf0aa1fb627c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:28 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:34562
I20260811 22:25:28.828033 26449 tablet_service.cc:1514] Processing DeleteTablet for tablet 886e8027a337401a8a2bf0aa1fb627c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:28 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:44902
I20260811 22:25:28.828060 26608 tablet_service.cc:1514] Processing DeleteTablet for tablet 886e8027a337401a8a2bf0aa1fb627c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:28 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:37704
I20260811 22:25:28.829033 26734 tablet_replica.cc:330] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: stopping tablet replica
I20260811 22:25:28.829409 26734 raft_consensus.cc:2229] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:28.829816 26734 raft_consensus.cc:2258] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:28.829842 26733 tablet_replica.cc:330] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: stopping tablet replica
I20260811 22:25:28.830142 26733 raft_consensus.cc:2229] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:28.830145 26137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:25:28.830528 26733 raft_consensus.cc:2258] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 22:25:28.830677 26137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:28.830857 26735 tablet_replica.cc:330] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: stopping tablet replica
I20260811 22:25:28.831164 26735 raft_consensus.cc:2229] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:28.831440 26735 raft_consensus.cc:2258] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:28.831858 26734 ts_tablet_manager.cc:1905] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:28.833261 26735 ts_tablet_manager.cc:1905] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:28.835286 26734 ts_tablet_manager.cc:1918] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:28.835415 26734 log.cc:1196] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/886e8027a337401a8a2bf0aa1fb627c3
I20260811 22:25:28.835801 26734 ts_tablet_manager.cc:1939] T 886e8027a337401a8a2bf0aa1fb627c3 P 9d59a980446d4750b59c93b02a859d0a: Deleting consensus metadata
I20260811 22:25:28.836306 26735 ts_tablet_manager.cc:1918] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:28.836410 26735 log.cc:1196] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/886e8027a337401a8a2bf0aa1fb627c3
I20260811 22:25:28.836720 26735 ts_tablet_manager.cc:1939] T 886e8027a337401a8a2bf0aa1fb627c3 P bfa42b46c70846c6a2c7d15401344a66: Deleting consensus metadata
I20260811 22:25:28.837513 26120 catalog_manager.cc:4841] TS 9d59a980446d4750b59c93b02a859d0a (127.24.71.194:36571): tablet 886e8027a337401a8a2bf0aa1fb627c3 (table test-table [id=a24d596cbb454aedad9d27d09981c46a]) successfully deleted
I20260811 22:25:28.838915 26733 ts_tablet_manager.cc:1905] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:28.839910 26608 tablet_service.cc:1467] Processing CreateTablet for tablet 1b237b4e0a0e442197fb2f0008393861 (DEFAULT_TABLE table=test-table [id=41b2e7ee3e744e32814ed415ddffa122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:28.839918 26292 tablet_service.cc:1467] Processing CreateTablet for tablet 1b237b4e0a0e442197fb2f0008393861 (DEFAULT_TABLE table=test-table [id=41b2e7ee3e744e32814ed415ddffa122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:28.840195 26292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b237b4e0a0e442197fb2f0008393861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.840195 26608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b237b4e0a0e442197fb2f0008393861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.842252 26733 ts_tablet_manager.cc:1918] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:28.842381 26733 log.cc:1196] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/886e8027a337401a8a2bf0aa1fb627c3
I20260811 22:25:28.842705 26710 tablet_bootstrap.cc:492] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: Bootstrap starting.
I20260811 22:25:28.842705 26709 tablet_bootstrap.cc:492] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: Bootstrap starting.
I20260811 22:25:28.842751 26733 ts_tablet_manager.cc:1939] T 886e8027a337401a8a2bf0aa1fb627c3 P 80b8b52ea83b4a628087e33c1a171932: Deleting consensus metadata
I20260811 22:25:28.843773 26709 tablet_bootstrap.cc:654] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.843772 26710 tablet_bootstrap.cc:654] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.844802 26119 catalog_manager.cc:4841] TS bfa42b46c70846c6a2c7d15401344a66 (127.24.71.193:45835): tablet 886e8027a337401a8a2bf0aa1fb627c3 (table test-table [id=a24d596cbb454aedad9d27d09981c46a]) successfully deleted
I20260811 22:25:28.845769 26449 tablet_service.cc:1467] Processing CreateTablet for tablet 1b237b4e0a0e442197fb2f0008393861 (DEFAULT_TABLE table=test-table [id=41b2e7ee3e744e32814ed415ddffa122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:28.846194 26449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b237b4e0a0e442197fb2f0008393861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.846374 26709 tablet_bootstrap.cc:492] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: No bootstrap required, opened a new log
I20260811 22:25:28.846468 26709 ts_tablet_manager.cc:1397] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:25:28.847019 26709 raft_consensus.cc:348] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.847142 26709 raft_consensus.cc:374] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.847175 26709 raft_consensus.cc:729] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa42b46c70846c6a2c7d15401344a66, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.847319 26709 consensus_queue.cc:260] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.847648 26709 ts_tablet_manager.cc:1428] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:28.849035 26713 tablet_bootstrap.cc:492] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: Bootstrap starting.
I20260811 22:25:28.850255 26713 tablet_bootstrap.cc:654] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.851375 26710 tablet_bootstrap.cc:492] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: No bootstrap required, opened a new log
I20260811 22:25:28.851534 26710 ts_tablet_manager.cc:1397] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260811 22:25:28.852157 26710 raft_consensus.cc:348] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.852303 26710 raft_consensus.cc:374] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.852344 26710 raft_consensus.cc:729] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b8b52ea83b4a628087e33c1a171932, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.854015 26713 tablet_bootstrap.cc:492] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: No bootstrap required, opened a new log
I20260811 22:25:28.854130 26713 ts_tablet_manager.cc:1397] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:25:28.854792 26713 raft_consensus.cc:348] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.854945 26713 raft_consensus.cc:374] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.854995 26713 raft_consensus.cc:729] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d59a980446d4750b59c93b02a859d0a, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.855157 26713 consensus_queue.cc:260] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.855474 26713 ts_tablet_manager.cc:1428] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:28.855713 26119 catalog_manager.cc:4841] TS 80b8b52ea83b4a628087e33c1a171932 (127.24.71.195:38441): tablet 886e8027a337401a8a2bf0aa1fb627c3 (table test-table [id=a24d596cbb454aedad9d27d09981c46a]) successfully deleted
I20260811 22:25:28.856500 26710 consensus_queue.cc:260] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.858506 26710 ts_tablet_manager.cc:1428] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
W20260811 22:25:28.900115 26686 tablet.cc:2369] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:25:28.911113 26524 tablet.cc:2369] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:28.919884 26714 raft_consensus.cc:482] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:28.920089 26714 raft_consensus.cc:504] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.920557 26714 leader_election.cc:288] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa42b46c70846c6a2c7d15401344a66 (127.24.71.193:45835), 9d59a980446d4750b59c93b02a859d0a (127.24.71.194:36571)
W20260811 22:25:28.920761 26367 tablet.cc:2369] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:28.921116 26471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b237b4e0a0e442197fb2f0008393861" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d59a980446d4750b59c93b02a859d0a" is_pre_election: true
I20260811 22:25:28.921233 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b237b4e0a0e442197fb2f0008393861" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa42b46c70846c6a2c7d15401344a66" is_pre_election: true
I20260811 22:25:28.921389 26315 raft_consensus.cc:2454] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 0.
I20260811 22:25:28.921275 26471 raft_consensus.cc:2454] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 0.
I20260811 22:25:28.921809 26551 leader_election.cc:302] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 80b8b52ea83b4a628087e33c1a171932, 9d59a980446d4750b59c93b02a859d0a, bfa42b46c70846c6a2c7d15401344a66; no voters:
I20260811 22:25:28.922089 26714 raft_consensus.cc:2781] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:28.922168 26714 raft_consensus.cc:482] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:28.922212 26714 raft_consensus.cc:3037] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:28.923532 26714 raft_consensus.cc:504] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.924185 26714 leader_election.cc:288] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 election: Requested vote from peers bfa42b46c70846c6a2c7d15401344a66 (127.24.71.193:45835), 9d59a980446d4750b59c93b02a859d0a (127.24.71.194:36571)
I20260811 22:25:28.924711 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b237b4e0a0e442197fb2f0008393861" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa42b46c70846c6a2c7d15401344a66"
I20260811 22:25:28.924702 26471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b237b4e0a0e442197fb2f0008393861" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d59a980446d4750b59c93b02a859d0a"
I20260811 22:25:28.924861 26315 raft_consensus.cc:3037] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:28.924860 26471 raft_consensus.cc:3037] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:28.926158 26471 raft_consensus.cc:2454] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 1.
I20260811 22:25:28.926158 26315 raft_consensus.cc:2454] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 1.
I20260811 22:25:28.926553 26550 leader_election.cc:302] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 80b8b52ea83b4a628087e33c1a171932, 9d59a980446d4750b59c93b02a859d0a, bfa42b46c70846c6a2c7d15401344a66; no voters:
I20260811 22:25:28.926738 26714 raft_consensus.cc:2781] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:28.926831 26714 raft_consensus.cc:686] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 1 LEADER]: Becoming Leader. State: Replica: 80b8b52ea83b4a628087e33c1a171932, State: Running, Role: LEADER
I20260811 22:25:28.927004 26714 consensus_queue.cc:237] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.929028 26137 catalog_manager.cc:5495] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80b8b52ea83b4a628087e33c1a171932 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } health_report { overall_health: HEALTHY } } }
Aug 11 22:25:28 dist-test-slave-mvvj krb5kdc[26085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487127, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:28.977061 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.966333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47630 (local address 127.24.71.254:38211)
0811 22:25:28.966471 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:28.966476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:28.966492 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:28.966646 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:28.966649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:28.966688 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:28.966773 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:28.967530 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.968131 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.968931 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.969140 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.969253 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.972552 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:28.972556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:28.972558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:28.972586 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:28.974547 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.975355 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.975358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.975360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.975501 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:28.975929 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:28.975933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:28.975937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:28.976169 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:28.976263 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:28.976742 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 22:25:28.976863 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:25:28.977072 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:28.965978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38211 (local address 127.0.0.1:47630)
0811 22:25:28.966353 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:28.966372 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:28.966516 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:28.966796 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:28.966799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:28.967287 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:28.967389 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.967404 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.968245 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.968250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.968804 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:28.968812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:28.969264 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:28.969267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:28.969346 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:28.970028 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:28.970042 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:28.972329 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:28.974715 (+ 2386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.974734 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.974744 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.975208 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.975623 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:28.975627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:28.975629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:28.975746 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:28.976308 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:28.976316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:28.976470 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 22:25:28.976614 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 22:25:28.976805 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":161,"threads_started":1}
I20260811 22:25:28.982651 26137 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:47630:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:25:28.983732 26137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:28.990272 26292 tablet_service.cc:1467] Processing CreateTablet for tablet fef858746b24490c8d2b3d78872202e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77fcabd16e984282b3914dcf9a8f96fd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:25:28.990444 26608 tablet_service.cc:1467] Processing CreateTablet for tablet fef858746b24490c8d2b3d78872202e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77fcabd16e984282b3914dcf9a8f96fd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:25:28.990859 26292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef858746b24490c8d2b3d78872202e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.990861 26608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef858746b24490c8d2b3d78872202e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.994238 26710 tablet_bootstrap.cc:492] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: Bootstrap starting.
I20260811 22:25:28.994349 26709 tablet_bootstrap.cc:492] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: Bootstrap starting.
I20260811 22:25:28.994735 26449 tablet_service.cc:1467] Processing CreateTablet for tablet fef858746b24490c8d2b3d78872202e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77fcabd16e984282b3914dcf9a8f96fd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:25:28.995126 26449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef858746b24490c8d2b3d78872202e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:28.995289 26710 tablet_bootstrap.cc:654] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.995582 26709 tablet_bootstrap.cc:654] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:28.997114 26709 tablet_bootstrap.cc:492] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: No bootstrap required, opened a new log
I20260811 22:25:28.997210 26709 ts_tablet_manager.cc:1397] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:25:28.997677 26709 raft_consensus.cc:348] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.997800 26709 raft_consensus.cc:374] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:28.997829 26709 raft_consensus.cc:729] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa42b46c70846c6a2c7d15401344a66, State: Initialized, Role: FOLLOWER
I20260811 22:25:28.997967 26709 consensus_queue.cc:260] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:28.998250 26709 ts_tablet_manager.cc:1428] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:25:28.998445 26713 tablet_bootstrap.cc:492] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: Bootstrap starting.
I20260811 22:25:28.998554 26715 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:28.998713 26715 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:28.999577 26713 tablet_bootstrap.cc:654] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:29.000723 26710 tablet_bootstrap.cc:492] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: No bootstrap required, opened a new log
I20260811 22:25:29.000821 26710 ts_tablet_manager.cc:1397] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260811 22:25:29.000916 26713 tablet_bootstrap.cc:492] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: No bootstrap required, opened a new log
I20260811 22:25:29.001006 26713 ts_tablet_manager.cc:1397] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:29.001247 26710 raft_consensus.cc:348] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:29.001358 26710 raft_consensus.cc:374] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:29.001386 26710 raft_consensus.cc:729] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b8b52ea83b4a628087e33c1a171932, State: Initialized, Role: FOLLOWER
I20260811 22:25:29.001507 26710 consensus_queue.cc:260] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:29.001506 26713 raft_consensus.cc:348] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:29.001624 26713 raft_consensus.cc:374] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:29.001662 26713 raft_consensus.cc:729] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d59a980446d4750b59c93b02a859d0a, State: Initialized, Role: FOLLOWER
I20260811 22:25:29.001784 26713 consensus_queue.cc:260] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:29.001780 26714 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:29.001932 26714 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:29.002072 26713 ts_tablet_manager.cc:1428] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:25:29.001792 26710 ts_tablet_manager.cc:1428] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:25:29.002272 26719 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:29.002408 26719 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:29.178614 26714 raft_consensus.cc:482] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:29.178785 26714 raft_consensus.cc:504] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:29.179262 26714 leader_election.cc:288] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa42b46c70846c6a2c7d15401344a66 (127.24.71.193:45835), 9d59a980446d4750b59c93b02a859d0a (127.24.71.194:36571)
I20260811 22:25:29.179889 26471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fef858746b24490c8d2b3d78872202e4" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d59a980446d4750b59c93b02a859d0a" is_pre_election: true
I20260811 22:25:29.179914 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fef858746b24490c8d2b3d78872202e4" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa42b46c70846c6a2c7d15401344a66" is_pre_election: true
I20260811 22:25:29.180056 26471 raft_consensus.cc:2454] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 0.
I20260811 22:25:29.180130 26315 raft_consensus.cc:2454] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 0.
I20260811 22:25:29.180466 26551 leader_election.cc:302] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80b8b52ea83b4a628087e33c1a171932, 9d59a980446d4750b59c93b02a859d0a; no voters:
I20260811 22:25:29.180722 26714 raft_consensus.cc:2781] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:29.180792 26714 raft_consensus.cc:482] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:29.180832 26714 raft_consensus.cc:3037] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:29.182229 26714 raft_consensus.cc:504] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:29.182605 26714 leader_election.cc:288] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 election: Requested vote from peers bfa42b46c70846c6a2c7d15401344a66 (127.24.71.193:45835), 9d59a980446d4750b59c93b02a859d0a (127.24.71.194:36571)
I20260811 22:25:29.183027 26471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fef858746b24490c8d2b3d78872202e4" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d59a980446d4750b59c93b02a859d0a"
I20260811 22:25:29.183030 26315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fef858746b24490c8d2b3d78872202e4" candidate_uuid: "80b8b52ea83b4a628087e33c1a171932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa42b46c70846c6a2c7d15401344a66"
I20260811 22:25:29.183166 26471 raft_consensus.cc:3037] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:29.183210 26315 raft_consensus.cc:3037] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:29.184283 26315 raft_consensus.cc:2454] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 1.
I20260811 22:25:29.184386 26471 raft_consensus.cc:2454] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80b8b52ea83b4a628087e33c1a171932 in term 1.
I20260811 22:25:29.184640 26550 leader_election.cc:302] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80b8b52ea83b4a628087e33c1a171932, bfa42b46c70846c6a2c7d15401344a66; no voters:
I20260811 22:25:29.184877 26714 raft_consensus.cc:2781] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:29.185002 26714 raft_consensus.cc:686] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [term 1 LEADER]: Becoming Leader. State: Replica: 80b8b52ea83b4a628087e33c1a171932, State: Running, Role: LEADER
I20260811 22:25:29.185163 26714 consensus_queue.cc:237] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } }
I20260811 22:25:29.185905 26716 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: TxnStatusTablet state changed. Reason: New leader 80b8b52ea83b4a628087e33c1a171932. Latest consensus state: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:29.186017 26716 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:25:29.186435 26751 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:25:29.186661 26751 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:25:29.187194 26135 catalog_manager.cc:5495] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80b8b52ea83b4a628087e33c1a171932 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } health_report { overall_health: HEALTHY } } }
I20260811 22:25:29.243155 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.238898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:38441 (local address 127.0.0.1:37710)
0811 22:25:29.239067 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:29.239088 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:29.239209 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:29.239724 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:29.239727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:29.239734 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:25:29.239903 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:29.239911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.241051 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.241054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:29.241728 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:29.241738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.242362 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.242365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:29.242475 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.242531 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:25:29.242866 (+ 335us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:25:29.242879 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 22:25:29.242940 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 22:25:29.243020 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 22:25:29.243175 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.239012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37710 (local address 127.24.71.195:38441)
0811 22:25:29.239166 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:29.239172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:29.239191 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:25:29.239470 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:29.239474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:29.239540 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:29.239645 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:29.240045 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.240908 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.241861 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.242240 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.242363 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.242680 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:29.242775 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:29.242855 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:29.242958 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 22:25:29.243020 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:25:29.246447 26471 raft_consensus.cc:1261] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a [term 1 FOLLOWER]: Refusing update from remote peer 80b8b52ea83b4a628087e33c1a171932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:29.246448 26315 raft_consensus.cc:1261] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Refusing update from remote peer 80b8b52ea83b4a628087e33c1a171932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:29.246997 26716 consensus_queue.cc:1035] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:29.247296 26714 consensus_queue.cc:1035] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:29.248891 26719 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: TxnStatusTablet state changed. Reason: New leader 80b8b52ea83b4a628087e33c1a171932. Latest consensus state: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:29.248889 26715 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: TxnStatusTablet state changed. Reason: New leader 80b8b52ea83b4a628087e33c1a171932. Latest consensus state: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:29.249025 26719 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:29.249032 26715 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:29.250885 26716 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:29.251015 26716 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:25:29.251669 26716 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:29.251780 26716 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P 80b8b52ea83b4a628087e33c1a171932: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:25:29.252408 26719 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:29.252556 26719 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P 9d59a980446d4750b59c93b02a859d0a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:29.255378 26715 tablet_replica.cc:439] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80b8b52ea83b4a628087e33c1a171932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 } } peers { permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 } } peers { permanent_uuid: "80b8b52ea83b4a628087e33c1a171932" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38441 } } }
I20260811 22:25:29.255519 26715 tablet_replica.cc:442] T fef858746b24490c8d2b3d78872202e4 P bfa42b46c70846c6a2c7d15401344a66: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:29.268941 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.265702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:42859 (local address 127.24.71.195:38441)
0811 22:25:29.265843 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:29.265848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:29.265863 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:25:29.266011 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:29.266017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:29.266071 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:29.266157 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:29.266431 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.266937 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.267796 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.268090 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.268200 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.268478 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:29.268554 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:29.268633 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:29.268716 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 22:25:29.268770 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":67}
I20260811 22:25:29.268951 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.265392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:38441 (local address 127.24.71.195:42859)
0811 22:25:29.265720 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:29.265768 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:29.265898 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:29.266166 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:29.266169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:29.266176 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:25:29.266305 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:29.266310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.267047 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.267053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:29.267674 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:29.267683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.268201 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.268205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:29.268288 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.268334 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:25:29.268635 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:25:29.268643 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:25:29.268708 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:25:29.268795 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":9,"thread_start_us":105,"threads_started":1}
I20260811 22:25:29.276772 26471 raft_consensus.cc:1261] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [term 1 FOLLOWER]: Refusing update from remote peer 80b8b52ea83b4a628087e33c1a171932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:29.276906 26315 raft_consensus.cc:1261] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Refusing update from remote peer 80b8b52ea83b4a628087e33c1a171932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:29.277249 26714 consensus_queue.cc:1035] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d59a980446d4750b59c93b02a859d0a" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 36571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:29.277524 26716 consensus_queue.cc:1035] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa42b46c70846c6a2c7d15401344a66" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 45835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:29.285490 26723 mvcc.cc:205] Tried to move back new op lower bound from 7317451281513926656 to 7317451280086437888. Current Snapshot: MvccSnapshot[applied={T|T < 7317451281513926656}]
I20260811 22:25:29.325951 26137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:29.326128 26137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:29.328588 26137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5fd1bc56364e48d49c34c28cf817b5f9: Sending DeleteTablet for 3 replicas of tablet 1b237b4e0a0e442197fb2f0008393861
I20260811 22:25:29.329407 26607 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b237b4e0a0e442197fb2f0008393861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:29 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:37704
I20260811 22:25:29.329645 26733 tablet_replica.cc:330] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: stopping tablet replica
I20260811 22:25:29.329636 26449 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b237b4e0a0e442197fb2f0008393861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:29 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:44902
I20260811 22:25:29.329785 26733 raft_consensus.cc:2229] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:29.329870 26734 tablet_replica.cc:330] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: stopping tablet replica
I20260811 22:25:29.330016 26734 raft_consensus.cc:2229] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:29.330041 26733 raft_consensus.cc:2258] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:29.330094 26292 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b237b4e0a0e442197fb2f0008393861 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:29 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:34562
I20260811 22:25:29.330202 26734 raft_consensus.cc:2258] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:29.331208 26734 ts_tablet_manager.cc:1905] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:29.331377 26735 tablet_replica.cc:330] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: stopping tablet replica
I20260811 22:25:29.331496 26735 raft_consensus.cc:2229] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:29.331625 26735 raft_consensus.cc:2258] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:29.332342 26733 ts_tablet_manager.cc:1905] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:29.334429 26734 ts_tablet_manager.cc:1918] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:25:29.334537 26734 log.cc:1196] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/1b237b4e0a0e442197fb2f0008393861
I20260811 22:25:29.334812 26734 ts_tablet_manager.cc:1939] T 1b237b4e0a0e442197fb2f0008393861 P 9d59a980446d4750b59c93b02a859d0a: Deleting consensus metadata
I20260811 22:25:29.335664 26735 ts_tablet_manager.cc:1905] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:29.335686 26733 ts_tablet_manager.cc:1918] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:25:29.335765 26733 log.cc:1196] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/1b237b4e0a0e442197fb2f0008393861
I20260811 22:25:29.336107 26733 ts_tablet_manager.cc:1939] T 1b237b4e0a0e442197fb2f0008393861 P 80b8b52ea83b4a628087e33c1a171932: Deleting consensus metadata
I20260811 22:25:29.336844 26119 catalog_manager.cc:4841] TS 80b8b52ea83b4a628087e33c1a171932 (127.24.71.195:38441): tablet 1b237b4e0a0e442197fb2f0008393861 (table test-table [id=41b2e7ee3e744e32814ed415ddffa122]) successfully deleted
I20260811 22:25:29.339080 26735 ts_tablet_manager.cc:1918] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:25:29.339179 26735 log.cc:1196] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/1b237b4e0a0e442197fb2f0008393861
I20260811 22:25:29.339488 26735 ts_tablet_manager.cc:1939] T 1b237b4e0a0e442197fb2f0008393861 P bfa42b46c70846c6a2c7d15401344a66: Deleting consensus metadata
I20260811 22:25:29.339958 26120 catalog_manager.cc:4841] TS 9d59a980446d4750b59c93b02a859d0a (127.24.71.194:36571): tablet 1b237b4e0a0e442197fb2f0008393861 (table test-table [id=41b2e7ee3e744e32814ed415ddffa122]) successfully deleted
I20260811 22:25:29.340683 26119 catalog_manager.cc:4841] TS bfa42b46c70846c6a2c7d15401344a66 (127.24.71.193:45835): tablet 1b237b4e0a0e442197fb2f0008393861 (table test-table [id=41b2e7ee3e744e32814ed415ddffa122]) successfully deleted
Aug 11 22:25:29 dist-test-slave-mvvj krb5kdc[26085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487128, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:25:29.350749 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.336985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.24.71.193:45835)
0811 22:25:29.337274 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:29.337280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:29.340548 (+ 3268us) server_negotiation.cc:407] Connection header received
0811 22:25:29.340761 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:29.340784 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:29.340844 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:29.340958 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:29.341669 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.342194 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.343062 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.343296 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.343411 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.346196 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:29.346214 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:29.346237 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:29.346270 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:29.348551 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:29.349102 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:29.349110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:29.349112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:29.349224 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:29.349553 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:29.349556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:29.349558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:29.349918 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:29.350158 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:29.350384 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 22:25:29.350534 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":126,"threads_started":1}
W20260811 22:25:29.351521 26360 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34568
I20260811 22:25:29.362444 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:29.353664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.24.71.254:38211)
0811 22:25:29.353840 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:29.353849 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:29.353900 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:25:29.354066 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:29.354069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:29.354132 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:29.354248 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:29.355127 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.356144 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.357023 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:29.357253 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:29.357387 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:29.359028 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:29.359031 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:29.359033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:29.359061 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:29.360556 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:29.361032 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:29.361036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:29.361038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:29.361128 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:29.361499 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:29.361503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:29.361505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:29.361711 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:29.361858 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:29.362111 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 22:25:29.362267 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 22:25:29.362999 26137 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47640
I20260811 22:25:29.364610 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 26219
I20260811 22:25:29.378712 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 26375
I20260811 22:25:29.388916 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 26533
I20260811 22:25:29.400342 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 26101
2026-08-11T22:25:29Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3067 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:29 dist-test-slave-mvvj krb5kdc[26776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:29 dist-test-slave-mvvj krb5kdc[26776](info): set up 2 sockets
Aug 11 22:25:29 dist-test-slave-mvvj krb5kdc[26776](info): commencing operation
krb5kdc: starting...
W20260811 22:25:30.559103 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:30 dist-test-slave-mvvj krb5kdc[26776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:30.709331 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:35943
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:35943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:30.835142 26790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:30.835469 26790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:30.835515 26790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:30.840035 26790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:30.840116 26790 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:25:30.840133 26790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:30.840152 26790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:30.840169 26790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:30.840185 26790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:30.845952 26790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:35943
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:35943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:30.847189 26790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:30.848318 26790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:30.858085 26795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:30.858175 26796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:30.858093 26800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:30.858701 26790 server_base.cc:1034] running on GCE node
I20260811 22:25:30.859335 26790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:30.860684 26790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:30.861903 26790 hybrid_clock.cc:648] HybridClock initialized: now 1786487130861886 us; error 31 us; skew 500 ppm
Aug 11 22:25:30 dist-test-slave-mvvj krb5kdc[26776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487130, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:30.868219 26790 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:30.869385 26790 webserver.cc:460] Webserver started at http://127.24.71.254:38507/ using document root <none> and password file <none>
I20260811 22:25:30.869987 26790 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:30.870054 26790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:30.870244 26790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:30.872560 26790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "91de33c4fc894e6abf98275b03f50782"
format_stamp: "Formatted at 2026-08-11 22:25:30 on dist-test-slave-mvvj"
server_key: "406532b7e91e0596e71849481c2a3765"
server_key_iv: "b89cd8220007e1587ea90ec8ed83b1f6"
server_key_version: "encryptionkey@0"
I20260811 22:25:30.873168 26790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "91de33c4fc894e6abf98275b03f50782"
format_stamp: "Formatted at 2026-08-11 22:25:30 on dist-test-slave-mvvj"
server_key: "406532b7e91e0596e71849481c2a3765"
server_key_iv: "b89cd8220007e1587ea90ec8ed83b1f6"
server_key_version: "encryptionkey@0"
I20260811 22:25:30.877477 26790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:30.880355 26808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:30.881839 26790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:30.882035 26790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "91de33c4fc894e6abf98275b03f50782"
format_stamp: "Formatted at 2026-08-11 22:25:30 on dist-test-slave-mvvj"
server_key: "406532b7e91e0596e71849481c2a3765"
server_key_iv: "b89cd8220007e1587ea90ec8ed83b1f6"
server_key_version: "encryptionkey@0"
I20260811 22:25:30.882172 26790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:30.912405 26790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:30.916100 26790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:30.916350 26790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:30.925407 26790 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:35943
I20260811 22:25:30.925452 26879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:35943 every 8 connection(s)
I20260811 22:25:30.926486 26790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:30.928582 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 26790
I20260811 22:25:30.928728 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:30.929050 24863 external_mini_cluster.cc:1428] Setting key 6a4f189dc3342fbccd32636236001d4f
I20260811 22:25:30.929767 26880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:30.936467 26880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782: Bootstrap starting.
Aug 11 22:25:30 dist-test-slave-mvvj krb5kdc[26776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:30.939437 26880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:30.940941 26880 log.cc:824] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:30.943281 26880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782: No bootstrap required, opened a new log
I20260811 22:25:30.944602 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:30.930531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.24.71.254:35943)
0811 22:25:30.931055 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:30.931073 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:30.931113 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:25:30.931864 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:30.931883 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:30.932295 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:30.932662 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:30.933577 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:30.934642 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:30.935553 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:30.936130 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:30.936315 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:30.939172 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:30.939179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:30.939200 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:30.939240 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:30.941469 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:30.941981 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:30.941992 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:30.941997 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:30.942121 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:30.942440 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:30.942443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:30.942445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:30.942879 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:30.943108 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:30.943435 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 22:25:30.943745 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":170,"threads_started":1}
I20260811 22:25:30.947279 26880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91de33c4fc894e6abf98275b03f50782" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35943 } }
I20260811 22:25:30.947569 26880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:30.947654 26880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91de33c4fc894e6abf98275b03f50782, State: Initialized, Role: FOLLOWER
I20260811 22:25:30.948210 26880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91de33c4fc894e6abf98275b03f50782" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35943 } }
I20260811 22:25:30.948372 26880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:30.948443 26880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:30.948570 26880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:30.949617 26880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91de33c4fc894e6abf98275b03f50782" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35943 } }
I20260811 22:25:30.949999 26880 leader_election.cc:302] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91de33c4fc894e6abf98275b03f50782; no voters:
I20260811 22:25:30.950433 26880 leader_election.cc:288] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:30.950752 26885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:30.951017 26885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [term 1 LEADER]: Becoming Leader. State: Replica: 91de33c4fc894e6abf98275b03f50782, State: Running, Role: LEADER
I20260811 22:25:30.951365 26885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91de33c4fc894e6abf98275b03f50782" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35943 } }
I20260811 22:25:30.952113 26880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:30.953418 26887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91de33c4fc894e6abf98275b03f50782. Latest consensus state: current_term: 1 leader_uuid: "91de33c4fc894e6abf98275b03f50782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91de33c4fc894e6abf98275b03f50782" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35943 } } }
I20260811 22:25:30.953565 26887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:30.953853 26886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91de33c4fc894e6abf98275b03f50782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91de33c4fc894e6abf98275b03f50782" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35943 } } }
I20260811 22:25:30.953966 26886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:30.953943 26895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:30.957574 26895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:30.964663 26895 catalog_manager.cc:1347] Generated new cluster ID: 48f1f9563ba84741914ac8e29c90a4e8
I20260811 22:25:30.964759 26895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:30.971256 26895 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:30.972506 26895 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:30.978121 26895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782: Generated new TSK 0
I20260811 22:25:30.979324 26895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:25:31.024866 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:35943
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:31.154193 26908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:31.154490 26908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:31.154541 26908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:31.158684 26908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:31.158767 26908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:31.158811 26908 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:25:31.158826 26908 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:25:31.158859 26908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:31.164615 26908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.71.254:35943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:31.165808 26908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:31.166875 26908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:31.177836 26913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.177845 26914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.177855 26916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:31.178248 26908 server_base.cc:1034] running on GCE node
I20260811 22:25:31.178771 26908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:31.179567 26908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:31.180850 26908 hybrid_clock.cc:648] HybridClock initialized: now 1786487131180802 us; error 71 us; skew 500 ppm
Aug 11 22:25:31 dist-test-slave-mvvj krb5kdc[26776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:31.184418 26908 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:25:31.185559 26908 webserver.cc:460] Webserver started at http://127.24.71.193:33609/ using document root <none> and password file <none>
I20260811 22:25:31.186244 26908 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:31.186347 26908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:31.186619 26908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:31.189113 26908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "2befb88ed8b648858a1abfb47307cd95"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-mvvj"
server_key: "408bd4afddd83fd3b0f77623af09b3eb"
server_key_iv: "bde44adba9237489748ef8017388fc07"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.189782 26908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "2befb88ed8b648858a1abfb47307cd95"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-mvvj"
server_key: "408bd4afddd83fd3b0f77623af09b3eb"
server_key_iv: "bde44adba9237489748ef8017388fc07"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.194351 26908 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:31.197822 26924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.199446 26908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:31.199743 26908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "2befb88ed8b648858a1abfb47307cd95"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-mvvj"
server_key: "408bd4afddd83fd3b0f77623af09b3eb"
server_key_iv: "bde44adba9237489748ef8017388fc07"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.200011 26908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:31.220165 26908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:31.223749 26908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:31.224020 26908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:31.225962 26908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:31.226029 26908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.226092 26908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:31.226125 26908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:25:31 dist-test-slave-mvvj krb5kdc[26776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:31.237920 26908 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:37433
I20260811 22:25:31.238016 27049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:37433 every 8 connection(s)
I20260811 22:25:31.238910 26908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:31.240689 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.226840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:48499 (local address 127.24.71.254:35943)
0811 22:25:31.227045 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.227052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.227992 (+ 940us) server_negotiation.cc:407] Connection header received
0811 22:25:31.228983 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.228987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.229050 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.229136 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.230546 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.231483 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.232903 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.233169 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.233290 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.236389 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.236393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.236396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.236423 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.238374 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.238998 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.239002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.239004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.239120 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.239532 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.239536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.239538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.239793 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.239934 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.240422 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.240563 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:25:31.241514 26933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.227264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35943 (local address 127.24.71.193:48499)
0811 22:25:31.227784 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.227873 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.228754 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.229441 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.229448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.229802 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.230367 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.230381 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.231623 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.231626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.232764 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.232775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.233350 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.233354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.233568 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.234251 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.234277 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.236184 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.238568 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.238575 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.238591 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.238872 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.239217 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.239221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.239223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.239403 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.239932 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.239938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.240176 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.240337 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.240609 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":116,"threads_started":1}
I20260811 22:25:31.243315 27050 heartbeater.cc:347] Connected to a master server at 127.24.71.254:35943
I20260811 22:25:31.243506 27050 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:31.243877 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 26908
I20260811 22:25:31.244016 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:31.244251 27050 heartbeater.cc:511] Master 127.24.71.254:35943 requested a full tablet report, sending...
I20260811 22:25:31.244421 24863 external_mini_cluster.cc:1428] Setting key 6aa1fe85f7f215f99add5c09852399c1
I20260811 22:25:31.246201 26820 ts_manager.cc:194] Registered new tserver with Master: 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433)
I20260811 22:25:31.247416 26820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:48499
I20260811 22:25:31.255348 27056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.245986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35943 (local address 127.24.71.193:46887)
0811 22:25:31.246392 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.246410 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.246519 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.248261 (+ 1742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.248264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.248488 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.248661 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.248668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.249456 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.249460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.250017 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.250026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.250525 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.250530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.250630 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.251206 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.251218 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.251593 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.253585 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.253590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.253592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.253885 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.254241 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.254244 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.254246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.254344 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.254758 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.254761 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.254869 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.254982 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.255160 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":127,"threads_started":1}
I20260811 22:25:31.255339 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.246362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:46887 (local address 127.24.71.254:35943)
0811 22:25:31.247993 (+ 1631us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.248000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.248019 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:25:31.248071 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.248076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.248134 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.248224 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.248790 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.249314 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.250152 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.250387 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.250496 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.251795 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.251798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.251800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.251824 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.253459 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.254021 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.254027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.254030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.254127 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.254442 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.254447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.254448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.254653 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.254768 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.255041 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.255170 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1522}
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:25:31.280511 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:35943
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:31.401793 27059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:31.402137 27059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:31.402194 27059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:31.406991 27059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:31.407094 27059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:31.407141 27059 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:25:31.407156 27059 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:25:31.407253 27059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:31.413606 27059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.71.254:35943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:31.415021 27059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:31.416221 27059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:31.427520 27059 server_base.cc:1034] running on GCE node
W20260811 22:25:31.427377 27067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.427405 27064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.427368 27065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:31.428311 27059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:31.429064 27059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:31.430299 27059 hybrid_clock.cc:648] HybridClock initialized: now 1786487131430235 us; error 71 us; skew 500 ppm
Aug 11 22:25:31 dist-test-slave-mvvj krb5kdc[26776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:31.433761 27059 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:25:31.434830 27059 webserver.cc:460] Webserver started at http://127.24.71.194:32907/ using document root <none> and password file <none>
I20260811 22:25:31.435472 27059 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:31.435573 27059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:31.435884 27059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:31.438139 27059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "7044e70d70b843ce9330f2a52556fb45"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-mvvj"
server_key: "e0328f4b09f5e7a35a3bba5d5a6bb78d"
server_key_iv: "176b5af35b8a0f6c7d7d411441cef8ff"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.438781 27059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "7044e70d70b843ce9330f2a52556fb45"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-mvvj"
server_key: "e0328f4b09f5e7a35a3bba5d5a6bb78d"
server_key_iv: "176b5af35b8a0f6c7d7d411441cef8ff"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.443221 27059 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:31.446384 27075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.448042 27059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:25:31.448199 27059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "7044e70d70b843ce9330f2a52556fb45"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-mvvj"
server_key: "e0328f4b09f5e7a35a3bba5d5a6bb78d"
server_key_iv: "176b5af35b8a0f6c7d7d411441cef8ff"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.448351 27059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:31.467989 27059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:31.471787 27059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:31.472080 27059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:31.473802 27059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:31.473886 27059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.473966 27059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:31.474009 27059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:25:31 dist-test-slave-mvvj krb5kdc[26776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:31.485805 27059 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:46137
I20260811 22:25:31.485874 27207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:46137 every 8 connection(s)
I20260811 22:25:31.487039 27059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:31.489181 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 27059
I20260811 22:25:31.489327 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:31.489674 24863 external_mini_cluster.cc:1428] Setting key ca18a56123dfcd897011907770419da7
I20260811 22:25:31.491627 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.474750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:54667 (local address 127.24.71.254:35943)
0811 22:25:31.474928 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.474933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.475920 (+ 987us) server_negotiation.cc:407] Connection header received
0811 22:25:31.477064 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.477071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.477171 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.477282 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.479380 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.480389 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.482281 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.482683 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.482804 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.486518 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.486557 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.486564 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.486606 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.488515 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.489110 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.489114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.489116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.489229 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.489710 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.489713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.489715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.490032 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.490200 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.491057 (+ 857us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.491233 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:25:31.492482 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.475233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35943 (local address 127.24.71.194:54667)
0811 22:25:31.475714 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.475770 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.476734 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.477636 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.477643 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.478224 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.479171 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.479194 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.480579 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.480583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.482131 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.482144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.483043 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.483050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.483296 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.484079 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.484108 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.486207 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.488637 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.488643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.488655 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.488981 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.489384 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.489391 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.489393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.489591 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.490225 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.490231 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.490764 (+ 533us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.490954 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.491292 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":137,"threads_started":1}
I20260811 22:25:31.495273 27208 heartbeater.cc:347] Connected to a master server at 127.24.71.254:35943
I20260811 22:25:31.495549 27208 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:31.496335 27208 heartbeater.cc:511] Master 127.24.71.254:35943 requested a full tablet report, sending...
I20260811 22:25:31.497828 26826 ts_manager.cc:194] Registered new tserver with Master: 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194:46137)
I20260811 22:25:31.498616 26826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:54667
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:25:31.508030 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.498353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35943 (local address 127.24.71.194:53393)
0811 22:25:31.498816 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.498835 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.498977 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.499245 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.499253 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.499522 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.499669 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.499677 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.501042 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.501046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.501672 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.501682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.502578 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.502581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.502679 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.503417 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.503431 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.503896 (+ 465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.506261 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.506264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.506266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.506552 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.506833 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.506835 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.506836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.506919 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.507447 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.507450 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.507539 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.507624 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.507752 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":130,"threads_started":1}
I20260811 22:25:31.508021 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.498901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:53393 (local address 127.24.71.254:35943)
0811 22:25:31.499017 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.499021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.499035 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:31.499074 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.499076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.499110 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.499221 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.499826 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.500857 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.501811 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.502280 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.502419 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.504157 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.504163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.504167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.504216 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.506147 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.506654 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.506657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.506658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.506745 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.507004 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.507006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.507007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.507312 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.507460 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.507685 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.507799 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:25:31.528235 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:35943
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:31.648494 27219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:31.648779 27219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:31.648830 27219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:31.653455 27219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:31.653529 27219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:31.653571 27219 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:25:31.653609 27219 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:25:31.653676 27219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:31.659545 27219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.71.254:35943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:31.660827 27219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:31.661968 27219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:31.673295 27226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:31.673295 27228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:31.673528 27219 server_base.cc:1034] running on GCE node
W20260811 22:25:31.673775 27225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:31.674268 27219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:31.675034 27219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:31.676240 27219 hybrid_clock.cc:648] HybridClock initialized: now 1786487131676210 us; error 44 us; skew 500 ppm
Aug 11 22:25:31 dist-test-slave-mvvj krb5kdc[26776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:31.679293 27219 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:25:31.680336 27219 webserver.cc:460] Webserver started at http://127.24.71.195:35087/ using document root <none> and password file <none>
I20260811 22:25:31.680909 27219 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:31.680984 27219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:31.681166 27219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:31.683334 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "c6beae24e06e46a6aad497f1ced74909"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-mvvj"
server_key: "ccdb54cebd137dc43d64e9e1d07cd5e7"
server_key_iv: "5e2bea4ce53dcbd5f26fbfe5925f3eff"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.683993 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "c6beae24e06e46a6aad497f1ced74909"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-mvvj"
server_key: "ccdb54cebd137dc43d64e9e1d07cd5e7"
server_key_iv: "5e2bea4ce53dcbd5f26fbfe5925f3eff"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.688663 27219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:25:31.691744 27235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.693394 27219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:25:31.693562 27219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "c6beae24e06e46a6aad497f1ced74909"
format_stamp: "Formatted at 2026-08-11 22:25:31 on dist-test-slave-mvvj"
server_key: "ccdb54cebd137dc43d64e9e1d07cd5e7"
server_key_iv: "5e2bea4ce53dcbd5f26fbfe5925f3eff"
server_key_version: "encryptionkey@0"
I20260811 22:25:31.693704 27219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:31.715137 27219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:31.718918 27219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:31.719161 27219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:31.721086 27219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:31.721148 27219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:31.721194 27219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:31.721266 27219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:25:31 dist-test-slave-mvvj krb5kdc[26776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:31.733063 27219 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:40673
I20260811 22:25:31.733131 27368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:40673 every 8 connection(s)
I20260811 22:25:31.734127 27219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:31.736624 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.722242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:48629 (local address 127.24.71.254:35943)
0811 22:25:31.722415 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.722420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.723333 (+ 913us) server_negotiation.cc:407] Connection header received
0811 22:25:31.724703 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.724708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.724774 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.724886 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.726134 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.726809 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.728437 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.728777 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.728906 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.732336 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.732340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.732343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.732379 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.734044 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.734612 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.734616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.734618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.734710 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.735157 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.735161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.735163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.735439 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.735562 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.736154 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.736297 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:25:31.736937 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 27219
I20260811 22:25:31.737087 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:31.737402 24863 external_mini_cluster.cc:1428] Setting key e6f17ee4973957ee174ec3cbfa56ffcd
I20260811 22:25:31.737666 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.722668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35943 (local address 127.24.71.195:48629)
0811 22:25:31.723161 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.723215 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.724455 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.725074 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.725093 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.725473 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.725966 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.725980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.726943 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.726948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.728287 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.728305 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.728973 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.728977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.729165 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.729915 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.729938 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.732146 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.734177 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.734183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.734198 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.734493 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.734854 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.734859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.734863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.735047 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.735561 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.735569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.735876 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.736085 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.736561 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":156,"threads_started":1}
I20260811 22:25:31.739269 27369 heartbeater.cc:347] Connected to a master server at 127.24.71.254:35943
I20260811 22:25:31.739470 27369 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:31.740206 27369 heartbeater.cc:511] Master 127.24.71.254:35943 requested a full tablet report, sending...
I20260811 22:25:31.741312 26820 ts_manager.cc:194] Registered new tserver with Master: c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673)
I20260811 22:25:31.742096 26820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:48629
I20260811 22:25:31.749511 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.741765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:52609 (local address 127.24.71.254:35943)
0811 22:25:31.741925 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.741932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.742095 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:25:31.742240 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.742243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.742299 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.742395 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.743005 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.743541 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.744659 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.744890 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.744984 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.746185 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.746188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.746191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.746216 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.747883 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.748379 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.748381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.748383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.748462 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.748746 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.748749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.748750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.748941 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.749022 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.749244 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.749392 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:25:31.749516 27375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.741708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35943 (local address 127.24.71.195:52609)
0811 22:25:31.741995 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.742011 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.742124 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.742414 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.742417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.742681 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.742836 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.742847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.743680 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.743683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.744520 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.744532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.745008 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.745011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.745114 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.745653 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.745665 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.745997 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.748010 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.748013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.748014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.748233 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.748564 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.748567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.748568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.748653 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.749030 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.749033 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.749121 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.749206 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.749323 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":117,"threads_started":1}
I20260811 22:25:31.752099 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:25:31 dist-test-slave-mvvj krb5kdc[26776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:25:31 dist-test-slave-mvvj krb5kdc[26776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:31.782150 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.772640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51324 (local address 127.24.71.254:35943)
0811 22:25:31.772788 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.772791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.772805 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:31.772852 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.772854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.772914 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.772984 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.773908 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.774644 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.775616 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.775918 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.776028 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.778755 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.778758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.778760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.778792 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.780165 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.780746 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.780749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.780750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.780838 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.781198 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.781202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.781203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.781486 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.781607 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.781888 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.782019 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:25:31.785969 26820 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:31.789407 26820 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:31.806445 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.800623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58180 (local address 127.24.71.193:37433)
0811 22:25:31.801005 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.801012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.801490 (+ 478us) server_negotiation.cc:407] Connection header received
0811 22:25:31.801582 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.801587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.801809 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.801977 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.802462 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.803468 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.805076 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.805743 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.805863 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.806038 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.806197 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.806301 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":106,"threads_started":1}
I20260811 22:25:31.806444 27386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.800069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37433 (local address 127.0.0.1:58180)
0811 22:25:31.801107 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.801166 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.801323 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.802009 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.802016 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.802054 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.802309 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.802317 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.803589 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.803594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.804941 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.804951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.805864 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.805867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.805987 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.806033 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.806148 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.806272 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"mutex_wait_us":32,"spinlock_wait_cycles":58624,"thread_start_us":105,"threads_started":1}
I20260811 22:25:31.807580 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.800131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46137 (local address 127.0.0.1:60922)
0811 22:25:31.801129 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.801169 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.802162 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.802736 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.802739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.802759 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.802866 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.802873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.804353 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.804357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.805598 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.805609 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.807030 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.807034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.807187 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.807223 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.807353 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.807436 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":96,"spinlock_wait_cycles":384,"thread_start_us":94,"threads_started":1}
I20260811 22:25:31.807633 27391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.800474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.24.71.194:46137)
0811 22:25:31.800874 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.800886 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.801262 (+ 376us) server_negotiation.cc:407] Connection header received
0811 22:25:31.802303 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.802308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.802503 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.802622 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.802993 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.804206 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.805739 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.806896 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.807029 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.807191 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.807389 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.807494 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":114,"threads_started":1}
I20260811 22:25:31.808244 27390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.799885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:40673 (local address 127.0.0.1:54134)
0811 22:25:31.801106 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.801164 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.801372 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.802013 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.802017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.802054 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.802777 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.802784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.804376 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.804379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.805683 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.805694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.806926 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.806929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.807063 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.807862 (+ 799us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.808026 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.808100 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":1285,"spinlock_wait_cycles":6784,"thread_start_us":90,"threads_started":1}
I20260811 22:25:31.808440 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.799965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.24.71.195:40673)
0811 22:25:31.801489 (+ 1524us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.801503 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.801524 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:25:31.801588 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.801592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.801809 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.801977 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.802905 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.804257 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.805784 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.806585 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.806692 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.806859 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.808146 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.808262 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1409,"thread_start_us":118,"threads_started":1}
I20260811 22:25:31.809999 26979 tablet_service.cc:1467] Processing CreateTablet for tablet af3cc8fae0dc4395b5bdba255d390dc4 (DEFAULT_TABLE table=test-table [id=d6f6e64729ab4eb7b7fe3f0b6e838a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:31.811251 26979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3cc8fae0dc4395b5bdba255d390dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:31.811292 27291 tablet_service.cc:1467] Processing CreateTablet for tablet af3cc8fae0dc4395b5bdba255d390dc4 (DEFAULT_TABLE table=test-table [id=d6f6e64729ab4eb7b7fe3f0b6e838a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:31.812464 27291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3cc8fae0dc4395b5bdba255d390dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:31.813208 27113 tablet_service.cc:1467] Processing CreateTablet for tablet af3cc8fae0dc4395b5bdba255d390dc4 (DEFAULT_TABLE table=test-table [id=d6f6e64729ab4eb7b7fe3f0b6e838a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:31.814332 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3cc8fae0dc4395b5bdba255d390dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:31.820068 27394 tablet_bootstrap.cc:492] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: Bootstrap starting.
I20260811 22:25:31.821393 27395 tablet_bootstrap.cc:492] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: Bootstrap starting.
I20260811 22:25:31.822603 27396 tablet_bootstrap.cc:492] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: Bootstrap starting.
I20260811 22:25:31.822898 27394 tablet_bootstrap.cc:654] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:31.823575 27395 tablet_bootstrap.cc:654] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:31.823904 27394 log.cc:824] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:31.824434 27395 log.cc:824] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:31.824855 27396 tablet_bootstrap.cc:654] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:31.825735 27396 log.cc:824] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:31.826459 27394 tablet_bootstrap.cc:492] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: No bootstrap required, opened a new log
I20260811 22:25:31.826682 27395 tablet_bootstrap.cc:492] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: No bootstrap required, opened a new log
I20260811 22:25:31.826705 27394 ts_tablet_manager.cc:1397] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260811 22:25:31.826894 27395 ts_tablet_manager.cc:1397] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:31.827862 27396 tablet_bootstrap.cc:492] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: No bootstrap required, opened a new log
I20260811 22:25:31.828076 27396 ts_tablet_manager.cc:1397] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:25:31.830260 27395 raft_consensus.cc:348] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } }
I20260811 22:25:31.830542 27395 raft_consensus.cc:374] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:31.830600 27395 raft_consensus.cc:729] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7044e70d70b843ce9330f2a52556fb45, State: Initialized, Role: FOLLOWER
I20260811 22:25:31.831085 27395 consensus_queue.cc:260] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } }
I20260811 22:25:31.831636 27394 raft_consensus.cc:348] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } }
I20260811 22:25:31.831878 27208 heartbeater.cc:503] Master 127.24.71.254:35943 was elected leader, sending a full tablet report...
I20260811 22:25:31.831948 27394 raft_consensus.cc:374] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:31.832028 27394 raft_consensus.cc:729] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2befb88ed8b648858a1abfb47307cd95, State: Initialized, Role: FOLLOWER
I20260811 22:25:31.832278 27395 ts_tablet_manager.cc:1428] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:25:31.832113 27396 raft_consensus.cc:348] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } }
I20260811 22:25:31.832409 27396 raft_consensus.cc:374] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:31.832461 27396 raft_consensus.cc:729] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6beae24e06e46a6aad497f1ced74909, State: Initialized, Role: FOLLOWER
I20260811 22:25:31.832492 27394 consensus_queue.cc:260] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } }
I20260811 22:25:31.832886 27396 consensus_queue.cc:260] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } }
I20260811 22:25:31.833524 27394 ts_tablet_manager.cc:1428] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260811 22:25:31.833619 27369 heartbeater.cc:503] Master 127.24.71.254:35943 was elected leader, sending a full tablet report...
I20260811 22:25:31.833860 27396 ts_tablet_manager.cc:1428] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:25:31.833999 27050 heartbeater.cc:503] Master 127.24.71.254:35943 was elected leader, sending a full tablet report...
I20260811 22:25:31.923952 27400 raft_consensus.cc:482] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:31.924252 27400 raft_consensus.cc:504] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } }
I20260811 22:25:31.925588 27400 leader_election.cc:288] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673), 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433)
I20260811 22:25:31.930497 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.925876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:40673 (local address 127.24.71.194:47825)
0811 22:25:31.926280 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.926298 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.926430 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.926722 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.926725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.926742 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.926866 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.926871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.927799 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.927805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.929099 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.929109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.930015 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.930019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.930152 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.930179 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.930265 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.930347 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":35}
I20260811 22:25:31.930519 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.926012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:47825 (local address 127.24.71.195:40673)
0811 22:25:31.926154 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.926160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.926366 (+ 206us) server_negotiation.cc:407] Connection header received
0811 22:25:31.926565 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.926572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.926615 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.926703 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.926996 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.927684 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.929244 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.929878 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.930004 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.930082 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.930304 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.930384 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:25:31.931131 27316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3cc8fae0dc4395b5bdba255d390dc4" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6beae24e06e46a6aad497f1ced74909" is_pre_election: true
I20260811 22:25:31.931393 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.926088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:43581 (local address 127.24.71.193:37433)
0811 22:25:31.926213 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.926218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.926540 (+ 322us) server_negotiation.cc:407] Connection header received
0811 22:25:31.926763 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.926774 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.926868 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.926917 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:31.927873 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.928665 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.930127 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.930743 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.930837 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.930891 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.931195 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.931283 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:25:31.931437 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.925878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37433 (local address 127.24.71.194:43581)
0811 22:25:31.926437 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.926453 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.926622 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.927050 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.927053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.927067 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:31.927694 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.927705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.928807 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.928811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.929984 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.929995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.930967 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.930971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.931072 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.931095 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.931171 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.931235 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":123,"thread_start_us":137,"threads_started":1}
I20260811 22:25:31.931484 27316 raft_consensus.cc:2454] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 0.
I20260811 22:25:31.932047 27080 leader_election.cc:302] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7044e70d70b843ce9330f2a52556fb45, c6beae24e06e46a6aad497f1ced74909; no voters:
I20260811 22:25:31.932230 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3cc8fae0dc4395b5bdba255d390dc4" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2befb88ed8b648858a1abfb47307cd95" is_pre_election: true
I20260811 22:25:31.932353 27400 raft_consensus.cc:2781] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:31.932448 27400 raft_consensus.cc:482] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:31.932520 27400 raft_consensus.cc:3037] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:31.932550 27002 raft_consensus.cc:2454] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 0.
I20260811 22:25:31.933753 27400 raft_consensus.cc:504] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } }
I20260811 22:25:31.934159 27400 leader_election.cc:288] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 election: Requested vote from peers c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673), 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433)
I20260811 22:25:31.934532 27316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3cc8fae0dc4395b5bdba255d390dc4" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6beae24e06e46a6aad497f1ced74909"
I20260811 22:25:31.934597 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3cc8fae0dc4395b5bdba255d390dc4" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2befb88ed8b648858a1abfb47307cd95"
I20260811 22:25:31.934700 27316 raft_consensus.cc:3037] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:31.934733 27002 raft_consensus.cc:3037] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:31.935879 27316 raft_consensus.cc:2454] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 1.
I20260811 22:25:31.936105 27002 raft_consensus.cc:2454] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 1.
I20260811 22:25:31.936213 27080 leader_election.cc:302] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7044e70d70b843ce9330f2a52556fb45, c6beae24e06e46a6aad497f1ced74909; no voters:
I20260811 22:25:31.936426 27400 raft_consensus.cc:2781] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:31.936713 27400 raft_consensus.cc:686] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 1 LEADER]: Becoming Leader. State: Replica: 7044e70d70b843ce9330f2a52556fb45, State: Running, Role: LEADER
I20260811 22:25:31.937090 27400 consensus_queue.cc:237] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } }
I20260811 22:25:31.940842 26826 catalog_manager.cc:5495] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194). New cstate: current_term: 1 leader_uuid: "7044e70d70b843ce9330f2a52556fb45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } health_report { overall_health: HEALTHY } } }
W20260811 22:25:31.985589 27370 tablet.cc:2369] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:25:31.988579 27210 tablet.cc:2369] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:25:31.990484 27051 tablet.cc:2369] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:25:31 dist-test-slave-mvvj krb5kdc[26776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487130, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:31.995558 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.985909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35943 (local address 127.0.0.1:51326)
0811 22:25:31.986300 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:31.986312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:31.986458 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:31.986718 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:31.986721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:31.987224 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:31.987362 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.987374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.988217 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.988220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.988827 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:31.988837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.989284 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.989288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:31.989369 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.989982 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:31.989995 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:31.991792 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:31.993714 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.993724 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.993730 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.994011 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.994318 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:31.994320 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:31.994322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:31.994409 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:31.994920 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:31.994934 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:31.995080 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 22:25:31.995185 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:25:31.995318 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":156,"threads_started":1}
I20260811 22:25:31.995647 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:31.985993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.24.71.254:35943)
0811 22:25:31.986157 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:31.986163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:31.986394 (+ 231us) server_negotiation.cc:407] Connection header received
0811 22:25:31.986561 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:31.986564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:31.986609 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:31.986695 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:31.987494 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.988100 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.988943 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:31.989157 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:31.989266 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:31.992017 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:31.992020 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:31.992022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:31.992045 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:31.993559 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.994132 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.994135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.994137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.994227 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:31.994527 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:31.994530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:31.994531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:31.994750 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:31.994850 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:31.995290 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 22:25:31.995501 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:25:32.000713 26826 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:51326:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:25:32.001575 26826 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:32.007434 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 5761d2cbda7b423e866ba91224c44597 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c7a745fb64524674b3c6107be1af29e5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:25:32.007540 27291 tablet_service.cc:1467] Processing CreateTablet for tablet 5761d2cbda7b423e866ba91224c44597 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c7a745fb64524674b3c6107be1af29e5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:25:32.007405 26979 tablet_service.cc:1467] Processing CreateTablet for tablet 5761d2cbda7b423e866ba91224c44597 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c7a745fb64524674b3c6107be1af29e5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:25:32.008323 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5761d2cbda7b423e866ba91224c44597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:32.008327 27291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5761d2cbda7b423e866ba91224c44597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:32.008458 26979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5761d2cbda7b423e866ba91224c44597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:32.011554 27394 tablet_bootstrap.cc:492] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: Bootstrap starting.
I20260811 22:25:32.011554 27396 tablet_bootstrap.cc:492] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: Bootstrap starting.
I20260811 22:25:32.012791 27394 tablet_bootstrap.cc:654] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:32.012791 27396 tablet_bootstrap.cc:654] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:32.013602 27395 tablet_bootstrap.cc:492] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: Bootstrap starting.
I20260811 22:25:32.014504 27396 tablet_bootstrap.cc:492] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: No bootstrap required, opened a new log
I20260811 22:25:32.014576 27396 ts_tablet_manager.cc:1397] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:32.014699 27395 tablet_bootstrap.cc:654] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:32.014946 27394 tablet_bootstrap.cc:492] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: No bootstrap required, opened a new log
I20260811 22:25:32.014966 27396 raft_consensus.cc:348] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.015033 27394 ts_tablet_manager.cc:1397] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:25:32.015058 27396 raft_consensus.cc:374] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:32.015079 27396 raft_consensus.cc:729] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6beae24e06e46a6aad497f1ced74909, State: Initialized, Role: FOLLOWER
I20260811 22:25:32.015183 27396 consensus_queue.cc:260] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.015461 27402 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.015611 27402 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:32.015641 27394 raft_consensus.cc:348] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.015790 27394 raft_consensus.cc:374] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:32.016264 27394 raft_consensus.cc:729] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2befb88ed8b648858a1abfb47307cd95, State: Initialized, Role: FOLLOWER
I20260811 22:25:32.016500 27394 consensus_queue.cc:260] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.016875 27394 ts_tablet_manager.cc:1428] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:25:32.016993 27395 tablet_bootstrap.cc:492] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: No bootstrap required, opened a new log
I20260811 22:25:32.017091 27395 ts_tablet_manager.cc:1397] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:25:32.017267 27396 ts_tablet_manager.cc:1428] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:32.017503 27401 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.017678 27401 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:32.017831 27395 raft_consensus.cc:348] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.017956 27395 raft_consensus.cc:374] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:32.017992 27395 raft_consensus.cc:729] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7044e70d70b843ce9330f2a52556fb45, State: Initialized, Role: FOLLOWER
I20260811 22:25:32.018122 27395 consensus_queue.cc:260] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.018468 27395 ts_tablet_manager.cc:1428] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:25:32.018718 27400 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.018879 27400 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:32.071916 27401 raft_consensus.cc:482] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:32.072268 27401 raft_consensus.cc:504] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.073642 27401 leader_election.cc:288] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194:46137), c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673)
I20260811 22:25:32.079031 27391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.074114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:40325 (local address 127.24.71.194:46137)
0811 22:25:32.074253 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.074258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.074682 (+ 424us) server_negotiation.cc:407] Connection header received
0811 22:25:32.074790 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.074793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.074856 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.074961 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:32.075487 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.076210 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.077709 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.078365 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.078469 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.078515 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.078806 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.078895 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:25:32.079064 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.074019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46137 (local address 127.24.71.193:40325)
0811 22:25:32.074472 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.074489 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.074659 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.075006 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.075010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.075039 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:32.075252 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.075265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.076361 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.076365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.077559 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.077570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.078551 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.078555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.078665 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.078692 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.078783 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.078860 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":37,"spinlock_wait_cycles":31104,"thread_start_us":196,"threads_started":1}
I20260811 22:25:32.079627 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.074114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:45495 (local address 127.24.71.195:40673)
0811 22:25:32.074253 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.074258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.074629 (+ 371us) server_negotiation.cc:407] Connection header received
0811 22:25:32.074799 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.074802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.074898 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.074979 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:32.076211 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.076988 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.078288 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.078947 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.079127 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.079181 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.079443 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.079522 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:25:32.079638 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.074019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:40673 (local address 127.24.71.193:45495)
0811 22:25:32.074495 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.074509 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.074659 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.075006 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.075010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.075062 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:32.076050 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.076065 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.077122 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.077124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.078073 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.078084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.079106 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.079111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.079214 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.079250 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.079412 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.079484 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":63,"spinlock_wait_cycles":78848,"thread_start_us":158,"threads_started":1}
I20260811 22:25:32.079862 27150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5761d2cbda7b423e866ba91224c44597" candidate_uuid: "2befb88ed8b648858a1abfb47307cd95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7044e70d70b843ce9330f2a52556fb45" is_pre_election: true
I20260811 22:25:32.080149 27150 raft_consensus.cc:2454] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2befb88ed8b648858a1abfb47307cd95 in term 0.
I20260811 22:25:32.080305 27316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5761d2cbda7b423e866ba91224c44597" candidate_uuid: "2befb88ed8b648858a1abfb47307cd95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6beae24e06e46a6aad497f1ced74909" is_pre_election: true
I20260811 22:25:32.080454 27316 raft_consensus.cc:2454] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2befb88ed8b648858a1abfb47307cd95 in term 0.
I20260811 22:25:32.080667 26927 leader_election.cc:302] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2befb88ed8b648858a1abfb47307cd95, 7044e70d70b843ce9330f2a52556fb45; no voters:
I20260811 22:25:32.081015 27401 raft_consensus.cc:2781] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:32.081079 27401 raft_consensus.cc:482] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:32.081106 27401 raft_consensus.cc:3037] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:32.082208 27401 raft_consensus.cc:504] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.082605 27401 leader_election.cc:288] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [CANDIDATE]: Term 1 election: Requested vote from peers 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194:46137), c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673)
I20260811 22:25:32.082991 27150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5761d2cbda7b423e866ba91224c44597" candidate_uuid: "2befb88ed8b648858a1abfb47307cd95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7044e70d70b843ce9330f2a52556fb45"
I20260811 22:25:32.083140 27150 raft_consensus.cc:3037] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:32.083185 27316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5761d2cbda7b423e866ba91224c44597" candidate_uuid: "2befb88ed8b648858a1abfb47307cd95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6beae24e06e46a6aad497f1ced74909"
I20260811 22:25:32.083323 27316 raft_consensus.cc:3037] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:32.084321 27150 raft_consensus.cc:2454] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2befb88ed8b648858a1abfb47307cd95 in term 1.
I20260811 22:25:32.084409 27316 raft_consensus.cc:2454] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2befb88ed8b648858a1abfb47307cd95 in term 1.
I20260811 22:25:32.084687 26927 leader_election.cc:302] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2befb88ed8b648858a1abfb47307cd95, 7044e70d70b843ce9330f2a52556fb45; no voters:
I20260811 22:25:32.084900 27401 raft_consensus.cc:2781] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:32.085166 27401 raft_consensus.cc:686] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [term 1 LEADER]: Becoming Leader. State: Replica: 2befb88ed8b648858a1abfb47307cd95, State: Running, Role: LEADER
I20260811 22:25:32.085623 27401 consensus_queue.cc:237] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.088344 27420 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: TxnStatusTablet state changed. Reason: New leader 2befb88ed8b648858a1abfb47307cd95. Latest consensus state: current_term: 1 leader_uuid: "2befb88ed8b648858a1abfb47307cd95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.088529 27420 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:25:32.088898 27422 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:25:32.089210 27422 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:25:32.089937 26826 catalog_manager.cc:5495] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193). New cstate: current_term: 1 leader_uuid: "2befb88ed8b648858a1abfb47307cd95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } health_report { overall_health: UNKNOWN } } }
I20260811 22:25:32.123723 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.118900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37433 (local address 127.0.0.1:58190)
0811 22:25:32.119250 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.119268 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.119396 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.119804 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.119806 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.119815 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:25:32.119998 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.120004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.121076 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.121079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.121723 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.121733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.122322 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.122325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.122439 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.122494 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:25:32.123371 (+ 877us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:25:32.123388 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.123475 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.123580 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236}
I20260811 22:25:32.123723 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.119042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58190 (local address 127.24.71.193:37433)
0811 22:25:32.119202 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.119207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.119355 (+ 148us) server_negotiation.cc:407] Connection header received
0811 22:25:32.119554 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.119557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.119634 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.119751 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:32.120124 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.120924 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.121853 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.122196 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.122303 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.122649 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:32.123234 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:32.123369 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.123488 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.123560 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:25:32.129714 27150 raft_consensus.cc:1261] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Refusing update from remote peer 2befb88ed8b648858a1abfb47307cd95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:32.129786 27316 raft_consensus.cc:1261] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Refusing update from remote peer 2befb88ed8b648858a1abfb47307cd95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:32.130481 27401 consensus_queue.cc:1035] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:32.130703 27420 consensus_queue.cc:1035] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:32.134150 27400 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: TxnStatusTablet state changed. Reason: New leader 2befb88ed8b648858a1abfb47307cd95. Latest consensus state: current_term: 1 leader_uuid: "2befb88ed8b648858a1abfb47307cd95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.134298 27400 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:32.135430 27402 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: TxnStatusTablet state changed. Reason: New leader 2befb88ed8b648858a1abfb47307cd95. Latest consensus state: current_term: 1 leader_uuid: "2befb88ed8b648858a1abfb47307cd95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.135576 27402 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:32.139217 27427 mvcc.cc:205] Tried to move back new op lower bound from 7317451293195771904 to 7317451293025546240. Current Snapshot: MvccSnapshot[applied={T|T < 7317451293195771904}]
I20260811 22:25:32.139369 27420 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2befb88ed8b648858a1abfb47307cd95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.139506 27420 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:25:32.139861 27400 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2befb88ed8b648858a1abfb47307cd95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.139986 27400 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P 7044e70d70b843ce9330f2a52556fb45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:32.142068 27401 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2befb88ed8b648858a1abfb47307cd95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.142194 27401 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P 2befb88ed8b648858a1abfb47307cd95: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:25:32.143529 27430 mvcc.cc:205] Tried to move back new op lower bound from 7317451293195771904 to 7317451293025546240. Current Snapshot: MvccSnapshot[applied={T|T < 7317451293195771904}]
I20260811 22:25:32.143970 27402 tablet_replica.cc:439] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2befb88ed8b648858a1abfb47307cd95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } } }
I20260811 22:25:32.144093 27402 tablet_replica.cc:442] T 5761d2cbda7b423e866ba91224c44597 P c6beae24e06e46a6aad497f1ced74909: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:25:32.152294 27391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.148142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.24.71.194:46137)
0811 22:25:32.148293 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.148297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.148321 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:25:32.148430 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.148433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.148499 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.148609 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:32.148861 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.149650 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.150490 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.150845 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.150953 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.151211 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:32.151806 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:32.151955 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.152089 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.152149 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:25:32.165247 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.160803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37433 (local address 127.24.71.194:58243)
0811 22:25:32.161187 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.161204 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.161337 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.161668 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.161672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.161681 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:25:32.161815 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.161822 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.162593 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.162597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.163633 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.163646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.164337 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.164342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.164444 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.164506 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:25:32.164887 (+ 381us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:25:32.164894 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.164965 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.165059 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":164,"threads_started":1}
I20260811 22:25:32.165235 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.160979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:58243 (local address 127.24.71.193:37433)
0811 22:25:32.161124 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.161128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.161282 (+ 154us) server_negotiation.cc:407] Connection header received
0811 22:25:32.161483 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.161485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.161534 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.161648 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:32.161928 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.162477 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.163782 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.164184 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.164276 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.164709 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:32.164790 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:32.164855 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.164996 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.165055 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:25:32.173620 27316 raft_consensus.cc:1261] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Refusing update from remote peer 7044e70d70b843ce9330f2a52556fb45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:32.173847 27002 raft_consensus.cc:1261] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Refusing update from remote peer 7044e70d70b843ce9330f2a52556fb45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:32.174155 27404 consensus_queue.cc:1035] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:32.174373 27400 consensus_queue.cc:1035] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:32.179786 27428 mvcc.cc:205] Tried to move back new op lower bound from 7317451293378269184 to 7317451292417425408. Current Snapshot: MvccSnapshot[applied={T|T < 7317451293378269184}]
I20260811 22:25:32.202009 27391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.197826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:42613 (local address 127.24.71.194:46137)
0811 22:25:32.197952 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:32.197958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:32.198230 (+ 272us) server_negotiation.cc:407] Connection header received
0811 22:25:32.198433 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:32.198436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:32.198496 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:32.198580 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:32.198886 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.199700 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.200637 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.200984 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.201101 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.201473 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:32.201560 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:32.201638 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:32.201778 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 22:25:32.201850 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 22:25:32.202042 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:32.197685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46137 (local address 127.24.71.193:42613)
0811 22:25:32.198131 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:32.198148 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:32.198265 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:32.198610 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:32.198613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:32.198622 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:25:32.198745 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.198751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.199857 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.199861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.200506 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:32.200515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:32.201137 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:32.201141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:32.201219 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:32.201275 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:25:32.201667 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:25:32.201675 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:25:32.201756 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:25:32.201853 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":165,"threads_started":1}
I20260811 22:25:32.240114 26826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:32.240326 26826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:32.243573 26826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782: Sending DeleteTablet for 3 replicas of tablet af3cc8fae0dc4395b5bdba255d390dc4
I20260811 22:25:32.244635 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet af3cc8fae0dc4395b5bdba255d390dc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:32 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60922
I20260811 22:25:32.244657 26979 tablet_service.cc:1514] Processing DeleteTablet for tablet af3cc8fae0dc4395b5bdba255d390dc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:32 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58180
I20260811 22:25:32.245559 27291 tablet_service.cc:1514] Processing DeleteTablet for tablet af3cc8fae0dc4395b5bdba255d390dc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:32 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:54134
I20260811 22:25:32.246928 26826 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:32.247437 26826 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:32.248929 27452 tablet_replica.cc:330] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: stopping tablet replica
I20260811 22:25:32.252986 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 4c224056e327473e93a9bfdfa5a07eb2 (DEFAULT_TABLE table=test-table [id=7192a9496dc44bbc963445920f82a0d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:32.253087 26979 tablet_service.cc:1467] Processing CreateTablet for tablet 4c224056e327473e93a9bfdfa5a07eb2 (DEFAULT_TABLE table=test-table [id=7192a9496dc44bbc963445920f82a0d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:32.253371 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c224056e327473e93a9bfdfa5a07eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:32.253391 26979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c224056e327473e93a9bfdfa5a07eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:32.256415 27395 tablet_bootstrap.cc:492] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: Bootstrap starting.
I20260811 22:25:32.257031 27291 tablet_service.cc:1467] Processing CreateTablet for tablet 4c224056e327473e93a9bfdfa5a07eb2 (DEFAULT_TABLE table=test-table [id=7192a9496dc44bbc963445920f82a0d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:32.257309 27291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c224056e327473e93a9bfdfa5a07eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:32.257499 27395 tablet_bootstrap.cc:654] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:32.261106 27394 tablet_bootstrap.cc:492] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: Bootstrap starting.
I20260811 22:25:32.261440 27451 tablet_replica.cc:330] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: stopping tablet replica
I20260811 22:25:32.262223 27394 tablet_bootstrap.cc:654] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:32.262681 27451 raft_consensus.cc:2229] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:32.263489 27451 raft_consensus.cc:2258] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:32.264155 27396 tablet_bootstrap.cc:492] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: Bootstrap starting.
I20260811 22:25:32.264885 27395 tablet_bootstrap.cc:492] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: No bootstrap required, opened a new log
I20260811 22:25:32.265022 27395 ts_tablet_manager.cc:1397] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260811 22:25:32.265347 27396 tablet_bootstrap.cc:654] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:32.266100 27395 raft_consensus.cc:348] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.266274 27395 raft_consensus.cc:374] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:32.266314 27395 raft_consensus.cc:729] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7044e70d70b843ce9330f2a52556fb45, State: Initialized, Role: FOLLOWER
I20260811 22:25:32.266633 27395 consensus_queue.cc:260] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.267006 27395 ts_tablet_manager.cc:1428] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:25:32.268988 27394 tablet_bootstrap.cc:492] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: No bootstrap required, opened a new log
I20260811 22:25:32.269126 27394 ts_tablet_manager.cc:1397] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260811 22:25:32.269747 27394 raft_consensus.cc:348] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.269896 27394 raft_consensus.cc:374] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:32.269935 27394 raft_consensus.cc:729] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2befb88ed8b648858a1abfb47307cd95, State: Initialized, Role: FOLLOWER
I20260811 22:25:32.270097 27394 consensus_queue.cc:260] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.270423 27394 ts_tablet_manager.cc:1428] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:32.275069 27451 ts_tablet_manager.cc:1905] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:32.276278 27452 raft_consensus.cc:2229] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:32.276638 27452 raft_consensus.cc:2258] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:32.277302 27450 tablet_replica.cc:330] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: stopping tablet replica
I20260811 22:25:32.277842 27450 raft_consensus.cc:2229] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:32.279228 27452 ts_tablet_manager.cc:1905] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:32.280010 27451 ts_tablet_manager.cc:1918] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:25:32.280061 27450 raft_consensus.cc:2258] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:32.280157 27451 log.cc:1196] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/af3cc8fae0dc4395b5bdba255d390dc4
I20260811 22:25:32.279783 27396 tablet_bootstrap.cc:492] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: No bootstrap required, opened a new log
I20260811 22:25:32.280577 27396 ts_tablet_manager.cc:1397] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20260811 22:25:32.280735 27451 ts_tablet_manager.cc:1939] T af3cc8fae0dc4395b5bdba255d390dc4 P 7044e70d70b843ce9330f2a52556fb45: Deleting consensus metadata
I20260811 22:25:32.281132 27396 raft_consensus.cc:348] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.281250 27396 raft_consensus.cc:374] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:32.281277 27396 raft_consensus.cc:729] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6beae24e06e46a6aad497f1ced74909, State: Initialized, Role: FOLLOWER
I20260811 22:25:32.281401 27396 consensus_queue.cc:260] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.281651 27396 ts_tablet_manager.cc:1428] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:32.284205 27450 ts_tablet_manager.cc:1905] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:32.284701 26811 catalog_manager.cc:4841] TS 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194:46137): tablet af3cc8fae0dc4395b5bdba255d390dc4 (table test-table [id=d6f6e64729ab4eb7b7fe3f0b6e838a60]) successfully deleted
I20260811 22:25:32.289366 27452 ts_tablet_manager.cc:1918] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:25:32.289639 27452 log.cc:1196] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/af3cc8fae0dc4395b5bdba255d390dc4
I20260811 22:25:32.290098 27452 ts_tablet_manager.cc:1939] T af3cc8fae0dc4395b5bdba255d390dc4 P c6beae24e06e46a6aad497f1ced74909: Deleting consensus metadata
I20260811 22:25:32.291697 27450 ts_tablet_manager.cc:1918] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:25:32.291853 27450 log.cc:1196] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/af3cc8fae0dc4395b5bdba255d390dc4
I20260811 22:25:32.291819 26812 catalog_manager.cc:4841] TS c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673): tablet af3cc8fae0dc4395b5bdba255d390dc4 (table test-table [id=d6f6e64729ab4eb7b7fe3f0b6e838a60]) successfully deleted
I20260811 22:25:32.292656 27450 ts_tablet_manager.cc:1939] T af3cc8fae0dc4395b5bdba255d390dc4 P 2befb88ed8b648858a1abfb47307cd95: Deleting consensus metadata
I20260811 22:25:32.296593 26809 catalog_manager.cc:4841] TS 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433): tablet af3cc8fae0dc4395b5bdba255d390dc4 (table test-table [id=d6f6e64729ab4eb7b7fe3f0b6e838a60]) successfully deleted
I20260811 22:25:32.348245 27404 raft_consensus.cc:482] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:32.348487 27404 raft_consensus.cc:504] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.349001 27404 leader_election.cc:288] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433), c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673)
I20260811 22:25:32.349459 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c224056e327473e93a9bfdfa5a07eb2" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2befb88ed8b648858a1abfb47307cd95" is_pre_election: true
I20260811 22:25:32.349488 27316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c224056e327473e93a9bfdfa5a07eb2" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6beae24e06e46a6aad497f1ced74909" is_pre_election: true
I20260811 22:25:32.349644 27002 raft_consensus.cc:2454] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 0.
I20260811 22:25:32.349663 27316 raft_consensus.cc:2454] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 0.
I20260811 22:25:32.349997 27077 leader_election.cc:302] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2befb88ed8b648858a1abfb47307cd95, 7044e70d70b843ce9330f2a52556fb45; no voters:
I20260811 22:25:32.350194 27404 raft_consensus.cc:2781] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:32.350286 27404 raft_consensus.cc:482] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:32.350351 27404 raft_consensus.cc:3037] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:32.351454 27404 raft_consensus.cc:504] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.351971 27404 leader_election.cc:288] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 election: Requested vote from peers 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433), c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673)
I20260811 22:25:32.352357 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c224056e327473e93a9bfdfa5a07eb2" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2befb88ed8b648858a1abfb47307cd95"
I20260811 22:25:32.352456 27316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c224056e327473e93a9bfdfa5a07eb2" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6beae24e06e46a6aad497f1ced74909"
I20260811 22:25:32.352528 27002 raft_consensus.cc:3037] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:32.352619 27316 raft_consensus.cc:3037] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:32.353889 27002 raft_consensus.cc:2454] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 1.
I20260811 22:25:32.353927 27316 raft_consensus.cc:2454] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 1.
I20260811 22:25:32.354355 27080 leader_election.cc:302] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2befb88ed8b648858a1abfb47307cd95, 7044e70d70b843ce9330f2a52556fb45, c6beae24e06e46a6aad497f1ced74909; no voters:
I20260811 22:25:32.354558 27404 raft_consensus.cc:2781] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:32.354660 27404 raft_consensus.cc:686] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 1 LEADER]: Becoming Leader. State: Replica: 7044e70d70b843ce9330f2a52556fb45, State: Running, Role: LEADER
I20260811 22:25:32.354812 27404 consensus_queue.cc:237] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:32.356904 26825 catalog_manager.cc:5495] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194). New cstate: current_term: 1 leader_uuid: "7044e70d70b843ce9330f2a52556fb45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } health_report { overall_health: UNKNOWN } } }
I20260811 22:25:32.385247 27316 raft_consensus.cc:1261] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Refusing update from remote peer 7044e70d70b843ce9330f2a52556fb45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:32.385545 27002 raft_consensus.cc:1261] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Refusing update from remote peer 7044e70d70b843ce9330f2a52556fb45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:32.385842 27445 consensus_queue.cc:1035] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:32.386081 27404 consensus_queue.cc:1035] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:25:32.403697 27428 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:25:33.963347 26826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:33.963514 26826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:33.966383 26826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782: Sending DeleteTablet for 3 replicas of tablet 4c224056e327473e93a9bfdfa5a07eb2
I20260811 22:25:33.966993 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c224056e327473e93a9bfdfa5a07eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:33 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60922
I20260811 22:25:33.966993 26979 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c224056e327473e93a9bfdfa5a07eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:33 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58180
I20260811 22:25:33.967222 27291 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c224056e327473e93a9bfdfa5a07eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:33 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:54134
I20260811 22:25:33.967393 27485 tablet_replica.cc:330] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: stopping tablet replica
I20260811 22:25:33.967604 27485 raft_consensus.cc:2229] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:33.967821 27485 raft_consensus.cc:2258] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:33.968152 27484 tablet_replica.cc:330] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: stopping tablet replica
I20260811 22:25:33.968255 27486 tablet_replica.cc:330] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: stopping tablet replica
I20260811 22:25:33.968291 27484 raft_consensus.cc:2229] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:33.968439 27486 raft_consensus.cc:2229] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:33.968585 27486 raft_consensus.cc:2258] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:33.968888 27485 ts_tablet_manager.cc:1905] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:33.969339 27486 ts_tablet_manager.cc:1905] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:33.971947 27485 ts_tablet_manager.cc:1918] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:25:33.972026 27484 raft_consensus.cc:2258] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:33.972083 27485 log.cc:1196] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/4c224056e327473e93a9bfdfa5a07eb2
I20260811 22:25:33.972674 27485 ts_tablet_manager.cc:1939] T 4c224056e327473e93a9bfdfa5a07eb2 P 7044e70d70b843ce9330f2a52556fb45: Deleting consensus metadata
I20260811 22:25:33.972787 27484 ts_tablet_manager.cc:1905] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:33.973281 27486 ts_tablet_manager.cc:1918] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:25:33.973446 27486 log.cc:1196] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/4c224056e327473e93a9bfdfa5a07eb2
I20260811 22:25:33.973531 26811 catalog_manager.cc:4841] TS 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194:46137): tablet 4c224056e327473e93a9bfdfa5a07eb2 (table test-table [id=7192a9496dc44bbc963445920f82a0d0]) successfully deleted
I20260811 22:25:33.973847 27486 ts_tablet_manager.cc:1939] T 4c224056e327473e93a9bfdfa5a07eb2 P c6beae24e06e46a6aad497f1ced74909: Deleting consensus metadata
I20260811 22:25:33.974768 26812 catalog_manager.cc:4841] TS c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673): tablet 4c224056e327473e93a9bfdfa5a07eb2 (table test-table [id=7192a9496dc44bbc963445920f82a0d0]) successfully deleted
I20260811 22:25:33.975728 27484 ts_tablet_manager.cc:1918] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:25:33.975826 27484 log.cc:1196] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/4c224056e327473e93a9bfdfa5a07eb2
I20260811 22:25:33.976161 27484 ts_tablet_manager.cc:1939] T 4c224056e327473e93a9bfdfa5a07eb2 P 2befb88ed8b648858a1abfb47307cd95: Deleting consensus metadata
I20260811 22:25:33.977116 26809 catalog_manager.cc:4841] TS 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433): tablet 4c224056e327473e93a9bfdfa5a07eb2 (table test-table [id=7192a9496dc44bbc963445920f82a0d0]) successfully deleted
Aug 11 22:25:33 dist-test-slave-mvvj krb5kdc[26776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:25:33 dist-test-slave-mvvj krb5kdc[26776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:34.001823 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:33.991182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.24.71.254:35943)
0811 22:25:33.991531 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:33.991542 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:33.991632 (+ 90us) server_negotiation.cc:407] Connection header received
0811 22:25:33.991855 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:33.991861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:33.991922 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:33.992024 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:33.992867 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:33.993487 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:33.994392 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:33.994640 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:33.994758 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:33.997947 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:33.997960 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:33.997963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:33.997996 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:33.999959 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:34.000495 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:34.000498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:34.000499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:34.000596 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:34.000914 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:34.000916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:34.000918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:34.001127 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:34.001241 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:34.001550 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 22:25:34.001674 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":122,"threads_started":1}
I20260811 22:25:34.005087 26826 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:34.005672 26826 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:34.011307 26979 tablet_service.cc:1467] Processing CreateTablet for tablet 33d928738be54912ba2802e5ca56f6b0 (DEFAULT_TABLE table=test-table [id=e6cc321af6b649608bf98c989e4f5117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:34.011369 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 33d928738be54912ba2802e5ca56f6b0 (DEFAULT_TABLE table=test-table [id=e6cc321af6b649608bf98c989e4f5117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:34.011586 27291 tablet_service.cc:1467] Processing CreateTablet for tablet 33d928738be54912ba2802e5ca56f6b0 (DEFAULT_TABLE table=test-table [id=e6cc321af6b649608bf98c989e4f5117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:34.011662 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d928738be54912ba2802e5ca56f6b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:34.011662 26979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d928738be54912ba2802e5ca56f6b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:34.011916 27291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d928738be54912ba2802e5ca56f6b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:34.014026 27497 tablet_bootstrap.cc:492] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: Bootstrap starting.
I20260811 22:25:34.014394 27498 tablet_bootstrap.cc:492] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: Bootstrap starting.
I20260811 22:25:34.015154 27499 tablet_bootstrap.cc:492] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: Bootstrap starting.
I20260811 22:25:34.015288 27498 tablet_bootstrap.cc:654] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:34.015534 27497 tablet_bootstrap.cc:654] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:34.016227 27499 tablet_bootstrap.cc:654] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:34.016683 27498 tablet_bootstrap.cc:492] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: No bootstrap required, opened a new log
I20260811 22:25:34.016795 27498 ts_tablet_manager.cc:1397] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:25:34.016968 27497 tablet_bootstrap.cc:492] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: No bootstrap required, opened a new log
I20260811 22:25:34.017066 27497 ts_tablet_manager.cc:1397] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:34.017371 27498 raft_consensus.cc:348] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.017532 27498 raft_consensus.cc:374] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:34.017572 27498 raft_consensus.cc:729] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7044e70d70b843ce9330f2a52556fb45, State: Initialized, Role: FOLLOWER
I20260811 22:25:34.017561 27497 raft_consensus.cc:348] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.017685 27497 raft_consensus.cc:374] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:34.017724 27497 raft_consensus.cc:729] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6beae24e06e46a6aad497f1ced74909, State: Initialized, Role: FOLLOWER
I20260811 22:25:34.017724 27498 consensus_queue.cc:260] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.017872 27497 consensus_queue.cc:260] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.018334 27497 ts_tablet_manager.cc:1428] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:25:34.018525 27498 ts_tablet_manager.cc:1428] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:25:34.019503 27499 tablet_bootstrap.cc:492] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: No bootstrap required, opened a new log
I20260811 22:25:34.019604 27499 ts_tablet_manager.cc:1397] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:25:34.020099 27499 raft_consensus.cc:348] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.020285 27499 raft_consensus.cc:374] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:34.020365 27499 raft_consensus.cc:729] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2befb88ed8b648858a1abfb47307cd95, State: Initialized, Role: FOLLOWER
I20260811 22:25:34.020542 27499 consensus_queue.cc:260] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.021523 27499 ts_tablet_manager.cc:1428] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:25:34.228300 27473 raft_consensus.cc:482] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:34.228543 27473 raft_consensus.cc:504] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.229077 27473 leader_election.cc:288] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433), c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673)
I20260811 22:25:34.229606 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d928738be54912ba2802e5ca56f6b0" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2befb88ed8b648858a1abfb47307cd95" is_pre_election: true
I20260811 22:25:34.229816 27002 raft_consensus.cc:2454] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 0.
I20260811 22:25:34.229892 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d928738be54912ba2802e5ca56f6b0" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6beae24e06e46a6aad497f1ced74909" is_pre_election: true
I20260811 22:25:34.230140 27314 raft_consensus.cc:2454] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 0.
I20260811 22:25:34.230293 27077 leader_election.cc:302] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2befb88ed8b648858a1abfb47307cd95, 7044e70d70b843ce9330f2a52556fb45; no voters:
I20260811 22:25:34.230537 27473 raft_consensus.cc:2781] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:34.230630 27473 raft_consensus.cc:482] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:34.230695 27473 raft_consensus.cc:3037] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:34.232016 27473 raft_consensus.cc:504] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.232462 27473 leader_election.cc:288] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 election: Requested vote from peers 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433), c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673)
I20260811 22:25:34.232944 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d928738be54912ba2802e5ca56f6b0" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2befb88ed8b648858a1abfb47307cd95"
I20260811 22:25:34.233006 27314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d928738be54912ba2802e5ca56f6b0" candidate_uuid: "7044e70d70b843ce9330f2a52556fb45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6beae24e06e46a6aad497f1ced74909"
I20260811 22:25:34.233110 27002 raft_consensus.cc:3037] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:34.233132 27314 raft_consensus.cc:3037] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:34.234467 27314 raft_consensus.cc:2454] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 1.
I20260811 22:25:34.234467 27002 raft_consensus.cc:2454] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7044e70d70b843ce9330f2a52556fb45 in term 1.
I20260811 22:25:34.234886 27080 leader_election.cc:302] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7044e70d70b843ce9330f2a52556fb45, c6beae24e06e46a6aad497f1ced74909; no voters:
I20260811 22:25:34.235116 27473 raft_consensus.cc:2781] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:34.235224 27473 raft_consensus.cc:686] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 1 LEADER]: Becoming Leader. State: Replica: 7044e70d70b843ce9330f2a52556fb45, State: Running, Role: LEADER
I20260811 22:25:34.235386 27473 consensus_queue.cc:237] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.237413 26825 catalog_manager.cc:5495] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194). New cstate: current_term: 1 leader_uuid: "7044e70d70b843ce9330f2a52556fb45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } health_report { overall_health: UNKNOWN } } }
I20260811 22:25:34.270726 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.267036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60936 (local address 127.24.71.194:46137)
0811 22:25:34.267383 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:34.267394 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:34.267408 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:34.267463 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:34.267465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:34.267516 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:34.267625 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:34.267991 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.268615 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.269522 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.269764 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.269885 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.270181 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:34.270262 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:34.270345 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:34.270494 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 22:25:34.270553 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":122,"threads_started":1}
I20260811 22:25:34.278621 27002 raft_consensus.cc:1261] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Refusing update from remote peer 7044e70d70b843ce9330f2a52556fb45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:34.278685 27314 raft_consensus.cc:1261] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Refusing update from remote peer 7044e70d70b843ce9330f2a52556fb45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:34.279055 27473 consensus_queue.cc:1035] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:34.279234 27476 consensus_queue.cc:1035] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:34.312059 26825 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:34.312263 26825 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:34.314651 26825 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782: Sending DeleteTablet for 3 replicas of tablet 33d928738be54912ba2802e5ca56f6b0
I20260811 22:25:34.315367 26979 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d928738be54912ba2802e5ca56f6b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:34 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58180
I20260811 22:25:34.315398 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d928738be54912ba2802e5ca56f6b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:34 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60922
I20260811 22:25:34.315639 27484 tablet_replica.cc:330] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: stopping tablet replica
I20260811 22:25:34.315733 27291 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d928738be54912ba2802e5ca56f6b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:34 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:54134
I20260811 22:25:34.315773 27484 raft_consensus.cc:2229] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:34.315932 27484 raft_consensus.cc:2258] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:34.315985 27486 tablet_replica.cc:330] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: stopping tablet replica
I20260811 22:25:34.316115 27486 raft_consensus.cc:2229] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:34.316129 27485 tablet_replica.cc:330] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: stopping tablet replica
I20260811 22:25:34.316233 27486 raft_consensus.cc:2258] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:34.316247 27485 raft_consensus.cc:2229] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:34.316471 27485 raft_consensus.cc:2258] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:34.316483 26825 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:34.316969 26825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:34.317203 27485 ts_tablet_manager.cc:1905] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:34.317330 27484 ts_tablet_manager.cc:1905] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:34.317355 27486 ts_tablet_manager.cc:1905] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:34.319794 27484 ts_tablet_manager.cc:1918] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:25:34.319947 27484 log.cc:1196] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/33d928738be54912ba2802e5ca56f6b0
I20260811 22:25:34.319985 27486 ts_tablet_manager.cc:1918] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:25:34.319985 27485 ts_tablet_manager.cc:1918] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:25:34.320082 27485 log.cc:1196] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/33d928738be54912ba2802e5ca56f6b0
I20260811 22:25:34.320081 27486 log.cc:1196] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/33d928738be54912ba2802e5ca56f6b0
I20260811 22:25:34.320250 27484 ts_tablet_manager.cc:1939] T 33d928738be54912ba2802e5ca56f6b0 P 2befb88ed8b648858a1abfb47307cd95: Deleting consensus metadata
I20260811 22:25:34.320364 27486 ts_tablet_manager.cc:1939] T 33d928738be54912ba2802e5ca56f6b0 P c6beae24e06e46a6aad497f1ced74909: Deleting consensus metadata
I20260811 22:25:34.320364 27485 ts_tablet_manager.cc:1939] T 33d928738be54912ba2802e5ca56f6b0 P 7044e70d70b843ce9330f2a52556fb45: Deleting consensus metadata
I20260811 22:25:34.321209 26809 catalog_manager.cc:4841] TS 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433): tablet 33d928738be54912ba2802e5ca56f6b0 (table test-table [id=e6cc321af6b649608bf98c989e4f5117]) successfully deleted
I20260811 22:25:34.321535 26811 catalog_manager.cc:4841] TS 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194:46137): tablet 33d928738be54912ba2802e5ca56f6b0 (table test-table [id=e6cc321af6b649608bf98c989e4f5117]) successfully deleted
I20260811 22:25:34.321642 26812 catalog_manager.cc:4841] TS c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673): tablet 33d928738be54912ba2802e5ca56f6b0 (table test-table [id=e6cc321af6b649608bf98c989e4f5117]) successfully deleted
I20260811 22:25:34.324693 26979 tablet_service.cc:1467] Processing CreateTablet for tablet 46d0c82282124b81b9c830bc780dfaa8 (DEFAULT_TABLE table=test-table [id=04b95d4dd8c345e194a7d01e0eb15997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:34.324893 27291 tablet_service.cc:1467] Processing CreateTablet for tablet 46d0c82282124b81b9c830bc780dfaa8 (DEFAULT_TABLE table=test-table [id=04b95d4dd8c345e194a7d01e0eb15997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:34.325060 26979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d0c82282124b81b9c830bc780dfaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:34.325158 27291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d0c82282124b81b9c830bc780dfaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:34.327649 27499 tablet_bootstrap.cc:492] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95: Bootstrap starting.
I20260811 22:25:34.327716 27497 tablet_bootstrap.cc:492] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: Bootstrap starting.
I20260811 22:25:34.328804 27497 tablet_bootstrap.cc:654] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:34.328809 27499 tablet_bootstrap.cc:654] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:34.329752 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 46d0c82282124b81b9c830bc780dfaa8 (DEFAULT_TABLE table=test-table [id=04b95d4dd8c345e194a7d01e0eb15997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:34.330017 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d0c82282124b81b9c830bc780dfaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:34.330019 27499 tablet_bootstrap.cc:492] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95: No bootstrap required, opened a new log
I20260811 22:25:34.330101 27499 ts_tablet_manager.cc:1397] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:34.330506 27499 raft_consensus.cc:348] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.330650 27499 raft_consensus.cc:374] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:34.330688 27499 raft_consensus.cc:729] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2befb88ed8b648858a1abfb47307cd95, State: Initialized, Role: FOLLOWER
I20260811 22:25:34.330837 27499 consensus_queue.cc:260] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.331197 27499 ts_tablet_manager.cc:1428] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:25:34.332377 27498 tablet_bootstrap.cc:492] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: Bootstrap starting.
I20260811 22:25:34.333559 27497 tablet_bootstrap.cc:492] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: No bootstrap required, opened a new log
I20260811 22:25:34.333582 27498 tablet_bootstrap.cc:654] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:34.333698 27497 ts_tablet_manager.cc:1397] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 22:25:34.334371 27497 raft_consensus.cc:348] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.334532 27497 raft_consensus.cc:374] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:34.334587 27497 raft_consensus.cc:729] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6beae24e06e46a6aad497f1ced74909, State: Initialized, Role: FOLLOWER
I20260811 22:25:34.334749 27497 consensus_queue.cc:260] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.335103 27498 tablet_bootstrap.cc:492] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: No bootstrap required, opened a new log
I20260811 22:25:34.335201 27498 ts_tablet_manager.cc:1397] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:34.335203 27497 ts_tablet_manager.cc:1428] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:25:34.335587 27498 raft_consensus.cc:348] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.335676 27498 raft_consensus.cc:374] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:34.335696 27498 raft_consensus.cc:729] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7044e70d70b843ce9330f2a52556fb45, State: Initialized, Role: FOLLOWER
I20260811 22:25:34.335793 27498 consensus_queue.cc:260] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.336112 27498 ts_tablet_manager.cc:1428] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:34.409693 27503 raft_consensus.cc:482] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:34.409909 27503 raft_consensus.cc:504] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.411185 27503 leader_election.cc:288] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433), 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194:46137)
I20260811 22:25:34.416002 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.411853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:60299 (local address 127.24.71.194:46137)
0811 22:25:34.412017 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:34.412020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:34.412171 (+ 151us) server_negotiation.cc:407] Connection header received
0811 22:25:34.412262 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:34.412265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:34.412316 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:34.412528 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:34.412826 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.413407 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.414798 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.415272 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.415422 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.415493 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:34.415736 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 22:25:34.415820 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:25:34.416049 27514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.411379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37433 (local address 127.24.71.195:37897)
0811 22:25:34.411660 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:34.411679 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:34.411867 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:34.412423 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:34.412427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:34.412450 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:34.412613 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:34.412625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.413465 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.413470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:34.414623 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:34.414635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.415482 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.415486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:34.415604 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.415624 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:25:34.415733 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:25:34.415816 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":77,"thread_start_us":111,"threads_started":1}
I20260811 22:25:34.416059 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.411561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:37897 (local address 127.24.71.193:37433)
0811 22:25:34.412124 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:34.412129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:34.412145 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:34.412204 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:34.412210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:34.412262 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:34.412351 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:34.412780 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.413330 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.414789 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.415302 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.415482 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.415546 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:34.415770 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 22:25:34.415894 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":99,"threads_started":1}
I20260811 22:25:34.416076 27517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.411602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46137 (local address 127.24.71.195:60299)
0811 22:25:34.412025 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:34.412041 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:34.412164 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:34.412540 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:34.412543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:34.412560 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:34.412671 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:34.412675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.413621 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.413624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:34.414621 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:34.414633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.415422 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.415429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:34.415571 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.415605 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:25:34.415691 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:25:34.415811 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":33,"thread_start_us":228,"threads_started":1}
I20260811 22:25:34.416672 27150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d0c82282124b81b9c830bc780dfaa8" candidate_uuid: "c6beae24e06e46a6aad497f1ced74909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7044e70d70b843ce9330f2a52556fb45" is_pre_election: true
I20260811 22:25:34.416672 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d0c82282124b81b9c830bc780dfaa8" candidate_uuid: "c6beae24e06e46a6aad497f1ced74909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2befb88ed8b648858a1abfb47307cd95" is_pre_election: true
I20260811 22:25:34.416849 27002 raft_consensus.cc:2454] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6beae24e06e46a6aad497f1ced74909 in term 0.
I20260811 22:25:34.416833 27150 raft_consensus.cc:2454] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6beae24e06e46a6aad497f1ced74909 in term 0.
I20260811 22:25:34.417385 27236 leader_election.cc:302] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2befb88ed8b648858a1abfb47307cd95, 7044e70d70b843ce9330f2a52556fb45, c6beae24e06e46a6aad497f1ced74909; no voters:
I20260811 22:25:34.417748 27503 raft_consensus.cc:2781] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:34.417819 27503 raft_consensus.cc:482] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:34.417851 27503 raft_consensus.cc:3037] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:34.419512 27503 raft_consensus.cc:504] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.420022 27503 leader_election.cc:288] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [CANDIDATE]: Term 1 election: Requested vote from peers 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433), 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194:46137)
I20260811 22:25:34.420549 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d0c82282124b81b9c830bc780dfaa8" candidate_uuid: "c6beae24e06e46a6aad497f1ced74909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2befb88ed8b648858a1abfb47307cd95"
I20260811 22:25:34.420565 27150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d0c82282124b81b9c830bc780dfaa8" candidate_uuid: "c6beae24e06e46a6aad497f1ced74909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7044e70d70b843ce9330f2a52556fb45"
I20260811 22:25:34.420712 27002 raft_consensus.cc:3037] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:34.420719 27150 raft_consensus.cc:3037] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:34.422060 27150 raft_consensus.cc:2454] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6beae24e06e46a6aad497f1ced74909 in term 1.
I20260811 22:25:34.422060 27002 raft_consensus.cc:2454] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6beae24e06e46a6aad497f1ced74909 in term 1.
I20260811 22:25:34.422458 27236 leader_election.cc:302] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2befb88ed8b648858a1abfb47307cd95, 7044e70d70b843ce9330f2a52556fb45, c6beae24e06e46a6aad497f1ced74909; no voters:
I20260811 22:25:34.422652 27503 raft_consensus.cc:2781] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:34.422966 27503 raft_consensus.cc:686] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 1 LEADER]: Becoming Leader. State: Replica: c6beae24e06e46a6aad497f1ced74909, State: Running, Role: LEADER
I20260811 22:25:34.423269 27503 consensus_queue.cc:237] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } }
I20260811 22:25:34.426222 26826 catalog_manager.cc:5495] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 reported cstate change: term changed from 0 to 1, leader changed from <none> to c6beae24e06e46a6aad497f1ced74909 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "c6beae24e06e46a6aad497f1ced74909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6beae24e06e46a6aad497f1ced74909" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40673 } health_report { overall_health: HEALTHY } } }
I20260811 22:25:34.467825 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.463439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54136 (local address 127.24.71.195:40673)
0811 22:25:34.463807 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:34.463820 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:34.463858 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:25:34.463951 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:34.463953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:34.464011 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:34.464126 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:34.464467 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.465031 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.465982 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.466250 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.466382 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.466750 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:34.467271 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:34.467373 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:34.467543 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:25:34.467603 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":161,"threads_started":1}
I20260811 22:25:34.480029 27523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.476102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37433 (local address 127.24.71.195:54543)
0811 22:25:34.476450 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:34.476464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:34.476570 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:34.476875 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:34.476877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:34.476885 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:25:34.477012 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:34.477023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.477919 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.477922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:34.478632 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:34.478643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.479206 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.479210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:34.479284 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.479339 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:25:34.479652 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:25:34.479657 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:25:34.479712 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 22:25:34.479784 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":129,"threads_started":1}
I20260811 22:25:34.480052 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.476241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:54543 (local address 127.24.71.193:37433)
0811 22:25:34.476413 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:34.476418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:34.476540 (+ 122us) server_negotiation.cc:407] Connection header received
0811 22:25:34.476663 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:34.476665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:34.476712 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:34.476788 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:34.477176 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.477760 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.478767 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.479050 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.479149 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.479488 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:34.479567 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:34.479628 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:34.479778 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 22:25:34.479929 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:25:34.486619 27150 raft_consensus.cc:1261] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Refusing update from remote peer c6beae24e06e46a6aad497f1ced74909: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:34.486781 27002 raft_consensus.cc:1261] T 46d0c82282124b81b9c830bc780dfaa8 P 2befb88ed8b648858a1abfb47307cd95 [term 1 FOLLOWER]: Refusing update from remote peer c6beae24e06e46a6aad497f1ced74909: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:34.487305 27503 consensus_queue.cc:1035] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7044e70d70b843ce9330f2a52556fb45" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:34.487546 27518 consensus_queue.cc:1035] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2befb88ed8b648858a1abfb47307cd95" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:34.510136 27529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.506303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:40673 (local address 127.24.71.193:59259)
0811 22:25:34.506604 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:34.506619 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:34.506722 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:34.506964 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:34.506966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:34.506975 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:25:34.507087 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:34.507093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.507875 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.507879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:34.508739 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:34.508750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.509382 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.509384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:34.509479 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.509514 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:25:34.509841 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:25:34.509848 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:25:34.509911 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 22:25:34.510004 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":133,"threads_started":1}
I20260811 22:25:34.510124 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:34.506617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:59259 (local address 127.24.71.195:40673)
0811 22:25:34.506733 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:34.506737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:34.506752 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:25:34.506805 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:34.506807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:34.506856 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:34.506950 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:34.507206 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.507731 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.508870 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:34.509257 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:34.509365 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:34.509651 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:34.509748 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:34.509820 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:34.509923 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 22:25:34.509978 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 22:25:34.511214 27464 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:25:36.088429 26826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:36.088688 26826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:36.091768 26826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91de33c4fc894e6abf98275b03f50782: Sending DeleteTablet for 3 replicas of tablet 46d0c82282124b81b9c830bc780dfaa8
I20260811 22:25:36.092530 27291 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d0c82282124b81b9c830bc780dfaa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:36 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:54134
I20260811 22:25:36.092537 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d0c82282124b81b9c830bc780dfaa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:36 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60922
I20260811 22:25:36.092640 26979 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d0c82282124b81b9c830bc780dfaa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:36 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58180
I20260811 22:25:36.093008 27558 tablet_replica.cc:330] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: stopping tablet replica
I20260811 22:25:36.093045 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 26908
I20260811 22:25:36.095106 27558 raft_consensus.cc:2229] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:36.095548 27558 raft_consensus.cc:2258] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:36.095914 27559 tablet_replica.cc:330] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: stopping tablet replica
I20260811 22:25:36.096131 27559 raft_consensus.cc:2229] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:36.096274 27559 raft_consensus.cc:2258] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:36.097039 27559 ts_tablet_manager.cc:1905] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:36.097054 27558 ts_tablet_manager.cc:1905] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:36.100117 27559 ts_tablet_manager.cc:1918] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:25:36.100265 27559 log.cc:1196] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/46d0c82282124b81b9c830bc780dfaa8
I20260811 22:25:36.100606 27559 ts_tablet_manager.cc:1939] T 46d0c82282124b81b9c830bc780dfaa8 P 7044e70d70b843ce9330f2a52556fb45: Deleting consensus metadata
I20260811 22:25:36.100716 27558 ts_tablet_manager.cc:1918] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:25:36.100811 27558 log.cc:1196] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/46d0c82282124b81b9c830bc780dfaa8
I20260811 22:25:36.101166 27558 ts_tablet_manager.cc:1939] T 46d0c82282124b81b9c830bc780dfaa8 P c6beae24e06e46a6aad497f1ced74909: Deleting consensus metadata
I20260811 22:25:36.101672 26811 catalog_manager.cc:4841] TS 7044e70d70b843ce9330f2a52556fb45 (127.24.71.194:46137): tablet 46d0c82282124b81b9c830bc780dfaa8 (table test-table [id=04b95d4dd8c345e194a7d01e0eb15997]) successfully deleted
I20260811 22:25:36.102070 26812 catalog_manager.cc:4841] TS c6beae24e06e46a6aad497f1ced74909 (127.24.71.195:40673): tablet 46d0c82282124b81b9c830bc780dfaa8 (table test-table [id=04b95d4dd8c345e194a7d01e0eb15997]) successfully deleted
W20260811 22:25:36.105233 26809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:37433 (error 108)
I20260811 22:25:36.106227 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 27059
I20260811 22:25:36.108577 27562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:36.106077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37433 (local address 127.0.0.1:58196)
0811 22:25:36.108239 (+ 2162us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:36.108385 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
W20260811 22:25:36.108762 26809 catalog_manager.cc:4584] TS 2befb88ed8b648858a1abfb47307cd95 (127.24.71.193:37433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 46d0c82282124b81b9c830bc780dfaa8: Network error: Client connection negotiation failed: client connection to 127.24.71.193:37433: connect: Connection refused (error 111)
I20260811 22:25:36.119380 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 27219
I20260811 22:25:36.130703 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 26790
2026-08-11T22:25:36Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6730 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:36 dist-test-slave-mvvj krb5kdc[27565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:36 dist-test-slave-mvvj krb5kdc[27565](info): set up 2 sockets
Aug 11 22:25:36 dist-test-slave-mvvj krb5kdc[27565](info): commencing operation
krb5kdc: starting...
W20260811 22:25:37.254465 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:37 dist-test-slave-mvvj krb5kdc[27565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:37.395908 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38427
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:38427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:37.523809 27583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:37.524173 27583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:37.524217 27583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:37.528368 27583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:37.528435 27583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:37.528455 27583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:37.528473 27583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:37.528488 27583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:37.534137 27583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:38427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:37.535276 27583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:37.536388 27583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:37.546286 27589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:37.546319 27591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:37.546285 27583 server_base.cc:1034] running on GCE node
W20260811 22:25:37.546262 27588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:37.547353 27583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:37.548684 27583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:37.549893 27583 hybrid_clock.cc:648] HybridClock initialized: now 1786487137549862 us; error 47 us; skew 500 ppm
Aug 11 22:25:37 dist-test-slave-mvvj krb5kdc[27565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487137, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:37.553105 27583 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:37.554072 27583 webserver.cc:460] Webserver started at http://127.24.71.254:46601/ using document root <none> and password file <none>
I20260811 22:25:37.554685 27583 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:37.554782 27583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:37.555047 27583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:37.557293 27583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "3e94a05963e142e7835c3bcbbb6fbd9d"
format_stamp: "Formatted at 2026-08-11 22:25:37 on dist-test-slave-mvvj"
server_key: "17dc3ef8473ea584f0ed58166f61a0bb"
server_key_iv: "d04aa472169593efba657c7760af0357"
server_key_version: "encryptionkey@0"
I20260811 22:25:37.557894 27583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "3e94a05963e142e7835c3bcbbb6fbd9d"
format_stamp: "Formatted at 2026-08-11 22:25:37 on dist-test-slave-mvvj"
server_key: "17dc3ef8473ea584f0ed58166f61a0bb"
server_key_iv: "d04aa472169593efba657c7760af0357"
server_key_version: "encryptionkey@0"
I20260811 22:25:37.562202 27583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:37.565080 27599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:37.566664 27583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:37.566843 27583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "3e94a05963e142e7835c3bcbbb6fbd9d"
format_stamp: "Formatted at 2026-08-11 22:25:37 on dist-test-slave-mvvj"
server_key: "17dc3ef8473ea584f0ed58166f61a0bb"
server_key_iv: "d04aa472169593efba657c7760af0357"
server_key_version: "encryptionkey@0"
I20260811 22:25:37.566989 27583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:37.593600 27583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:37.597858 27583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:37.598121 27583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:37.606673 27583 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:38427
I20260811 22:25:37.606766 27661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:38427 every 8 connection(s)
I20260811 22:25:37.607810 27583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:37.610904 27662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:37.614553 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 27583
I20260811 22:25:37.614692 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:37.614974 24863 external_mini_cluster.cc:1428] Setting key 3df614d26d148faedac7723c454b8a91
I20260811 22:25:37.617664 27662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d: Bootstrap starting.
I20260811 22:25:37.620579 27662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:37.621613 27662 log.cc:824] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:37.624550 27662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d: No bootstrap required, opened a new log
Aug 11 22:25:37 dist-test-slave-mvvj krb5kdc[27565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:37.627911 27662 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e94a05963e142e7835c3bcbbb6fbd9d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38427 } }
I20260811 22:25:37.628212 27662 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:37.628297 27662 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e94a05963e142e7835c3bcbbb6fbd9d, State: Initialized, Role: FOLLOWER
I20260811 22:25:37.628773 27662 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e94a05963e142e7835c3bcbbb6fbd9d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38427 } }
I20260811 22:25:37.628932 27662 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:37.629024 27662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:37.629158 27662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:37.630218 27662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e94a05963e142e7835c3bcbbb6fbd9d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38427 } }
I20260811 22:25:37.630594 27662 leader_election.cc:302] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e94a05963e142e7835c3bcbbb6fbd9d; no voters:
I20260811 22:25:37.631072 27662 leader_election.cc:288] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:37.631153 27667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:37.631479 27667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [term 1 LEADER]: Becoming Leader. State: Replica: 3e94a05963e142e7835c3bcbbb6fbd9d, State: Running, Role: LEADER
I20260811 22:25:37.631913 27667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e94a05963e142e7835c3bcbbb6fbd9d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38427 } }
I20260811 22:25:37.632256 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:37.616593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.24.71.254:38427)
0811 22:25:37.617091 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:37.617109 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:37.617148 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:25:37.617867 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:37.617887 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:37.618241 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:37.618545 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:37.619562 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:37.620654 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:37.621886 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:37.622591 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:37.622717 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:37.625782 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:37.625787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:37.625803 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:37.625832 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:37.628717 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:37.629259 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:37.629265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:37.629271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:37.629402 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:37.629725 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:37.629729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:37.629731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:37.630173 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:37.630405 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:37.630709 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 22:25:37.631074 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":150,"threads_started":1}
I20260811 22:25:37.632633 27662 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:37.633229 27667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e94a05963e142e7835c3bcbbb6fbd9d. Latest consensus state: current_term: 1 leader_uuid: "3e94a05963e142e7835c3bcbbb6fbd9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e94a05963e142e7835c3bcbbb6fbd9d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38427 } } }
I20260811 22:25:37.633436 27667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:37.633751 27668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e94a05963e142e7835c3bcbbb6fbd9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e94a05963e142e7835c3bcbbb6fbd9d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38427 } } }
I20260811 22:25:37.633868 27668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:37.634682 27673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:37.637759 27673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:37.643811 27673 catalog_manager.cc:1347] Generated new cluster ID: 2a1a0f8d3ff943f29fe75edb6205fac5
I20260811 22:25:37.643934 27673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:37.676124 27673 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:37.677269 27673 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:37.691229 27673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e94a05963e142e7835c3bcbbb6fbd9d: Generated new TSK 0
I20260811 22:25:37.692144 27673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:25:37.741531 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38427
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:37.869151 27692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:37.869465 27692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:37.869513 27692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:37.874006 27692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:37.874085 27692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:37.874148 27692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:37.879671 27692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:37.880915 27692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:37.882026 27692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:37.893168 27697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:37.893128 27698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:37.893332 27692 server_base.cc:1034] running on GCE node
W20260811 22:25:37.893128 27700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:37.894073 27692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:37.894820 27692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:37.896014 27692 hybrid_clock.cc:648] HybridClock initialized: now 1786487137895992 us; error 35 us; skew 500 ppm
Aug 11 22:25:37 dist-test-slave-mvvj krb5kdc[27565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487137, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:37.899312 27692 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:25:37.900588 27692 webserver.cc:460] Webserver started at http://127.24.71.193:38787/ using document root <none> and password file <none>
I20260811 22:25:37.901254 27692 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:37.901355 27692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:37.901605 27692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:37.903905 27692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "ffb3b3a897eb4f999f5055365775fcd2"
format_stamp: "Formatted at 2026-08-11 22:25:37 on dist-test-slave-mvvj"
server_key: "b74a4780b69e36d383913298ce56a572"
server_key_iv: "5b81b20c233f3a65c3c665b9804a1d26"
server_key_version: "encryptionkey@0"
I20260811 22:25:37.904557 27692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "ffb3b3a897eb4f999f5055365775fcd2"
format_stamp: "Formatted at 2026-08-11 22:25:37 on dist-test-slave-mvvj"
server_key: "b74a4780b69e36d383913298ce56a572"
server_key_iv: "5b81b20c233f3a65c3c665b9804a1d26"
server_key_version: "encryptionkey@0"
I20260811 22:25:37.908703 27692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:37.911280 27708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:37.912690 27692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:37.912844 27692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "ffb3b3a897eb4f999f5055365775fcd2"
format_stamp: "Formatted at 2026-08-11 22:25:37 on dist-test-slave-mvvj"
server_key: "b74a4780b69e36d383913298ce56a572"
server_key_iv: "5b81b20c233f3a65c3c665b9804a1d26"
server_key_version: "encryptionkey@0"
I20260811 22:25:37.912969 27692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:37.942250 27692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:37.945894 27692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:37.946128 27692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:37.946832 27692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:37.947739 27692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:37.947789 27692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:37.947876 27692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:37.947932 27692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:37.957568 27692 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:38213
I20260811 22:25:37.957609 27832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:38213 every 8 connection(s)
I20260811 22:25:37.958595 27692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:37.961565 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 27692
I20260811 22:25:37.961715 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:37.962033 24863 external_mini_cluster.cc:1428] Setting key 9d606daa9cb41cf9a9bb18b2e47c8f58
Aug 11 22:25:37 dist-test-slave-mvvj krb5kdc[27565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487137, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:25:37.975116 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:37.960573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:58081 (local address 127.24.71.254:38427)
0811 22:25:37.960739 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:37.960744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:37.962394 (+ 1650us) server_negotiation.cc:407] Connection header received
0811 22:25:37.963686 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:37.963689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:37.963753 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:37.963898 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:37.965246 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:37.965792 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:37.967009 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:37.967246 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:37.967355 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:37.970583 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:37.970586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:37.970588 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:37.970617 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:37.972503 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:37.973136 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:37.973141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:37.973143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:37.973249 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:37.973656 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:37.973660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:37.973662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:37.973897 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:37.974021 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:37.974732 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 22:25:37.974926 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:25:37.976295 27835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:37.961082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38427 (local address 127.24.71.193:58081)
0811 22:25:37.962196 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:37.962257 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:37.963405 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:37.964112 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:37.964121 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:37.964570 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:37.965086 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:37.965100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:37.965937 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:37.965941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:37.966871 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:37.966881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:37.967428 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:37.967433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:37.967608 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:37.968417 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:37.968437 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:37.970363 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:37.972656 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:37.972663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:37.972683 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:37.973007 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:37.973356 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:37.973359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:37.973362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:37.973536 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:37.974021 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:37.974030 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:37.974405 (+ 375us) client_negotiation.cc:769] Sending connection context
0811 22:25:37.974678 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 22:25:37.975123 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"thread_start_us":737,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
I20260811 22:25:37.977831 27833 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38427
I20260811 22:25:37.978134 27833 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:37.978777 27833 heartbeater.cc:511] Master 127.24.71.254:38427 requested a full tablet report, sending...
I20260811 22:25:37.980748 27622 ts_manager.cc:194] Registered new tserver with Master: ffb3b3a897eb4f999f5055365775fcd2 (127.24.71.193:38213)
I20260811 22:25:37.982509 27622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:58081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:25:37.999080 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38427
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:38.123750 27838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:38.124081 27838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:38.124128 27838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:38.128225 27838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:38.128294 27838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:38.128355 27838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:38.134228 27838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:38.135501 27838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:38.136577 27838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:38.147722 27846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:38.147719 27844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:38.148092 27838 server_base.cc:1034] running on GCE node
W20260811 22:25:38.147722 27843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:38.148650 27838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:38.149443 27838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:38.150652 27838 hybrid_clock.cc:648] HybridClock initialized: now 1786487138150631 us; error 32 us; skew 500 ppm
Aug 11 22:25:38 dist-test-slave-mvvj krb5kdc[27565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487138, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:38.153671 27838 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:25:38.154680 27838 webserver.cc:460] Webserver started at http://127.24.71.194:44087/ using document root <none> and password file <none>
I20260811 22:25:38.155293 27838 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:38.155361 27838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:38.155548 27838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:38.157807 27838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "61dc4db0d7a54008a86d4ae7a48f5bba"
format_stamp: "Formatted at 2026-08-11 22:25:38 on dist-test-slave-mvvj"
server_key: "9fe39912406a4c1a4affbe7792894aac"
server_key_iv: "eac446fe54236cc6f04bbd3f508213e9"
server_key_version: "encryptionkey@0"
I20260811 22:25:38.158450 27838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "61dc4db0d7a54008a86d4ae7a48f5bba"
format_stamp: "Formatted at 2026-08-11 22:25:38 on dist-test-slave-mvvj"
server_key: "9fe39912406a4c1a4affbe7792894aac"
server_key_iv: "eac446fe54236cc6f04bbd3f508213e9"
server_key_version: "encryptionkey@0"
I20260811 22:25:38.162667 27838 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:25:38.165639 27853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:38.167173 27838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:38.167307 27838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "61dc4db0d7a54008a86d4ae7a48f5bba"
format_stamp: "Formatted at 2026-08-11 22:25:38 on dist-test-slave-mvvj"
server_key: "9fe39912406a4c1a4affbe7792894aac"
server_key_iv: "eac446fe54236cc6f04bbd3f508213e9"
server_key_version: "encryptionkey@0"
I20260811 22:25:38.167482 27838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:38.181347 27838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:38.185424 27838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:38.185635 27838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:38.186352 27838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:38.187322 27838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:38.187378 27838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:38.187427 27838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:38.187469 27838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:38.198346 27838 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:46023
I20260811 22:25:38.198446 27986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:46023 every 8 connection(s)
I20260811 22:25:38.199394 27838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:38.207967 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 27838
I20260811 22:25:38.208112 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:38.208495 24863 external_mini_cluster.cc:1428] Setting key b5c9b3386a40663060d5945db8a36086
Aug 11 22:25:38 dist-test-slave-mvvj krb5kdc[27565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487138, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:38.214241 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:38.201711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:47123 (local address 127.24.71.254:38427)
0811 22:25:38.201855 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:38.201858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:38.202941 (+ 1083us) server_negotiation.cc:407] Connection header received
0811 22:25:38.203942 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:38.203951 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:38.204007 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:38.204097 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:38.205518 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:38.206153 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:38.207408 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:38.207667 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:38.207769 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:38.210589 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:38.210592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:38.210594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:38.210618 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:38.212129 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:38.212662 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:38.212666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:38.212668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:38.212749 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:38.213107 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:38.213111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:38.213113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:38.213306 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:38.213415 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:38.213944 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 22:25:38.214075 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:25:38.215216 27989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:38.202055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38427 (local address 127.24.71.194:47123)
0811 22:25:38.202769 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:38.202811 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:38.203650 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:38.204368 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:38.204376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:38.204799 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:38.205351 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:38.205366 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:38.206301 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:38.206304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:38.207270 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:38.207281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:38.207882 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:38.207888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:38.208129 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:38.208862 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:38.208881 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:38.210402 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:38.212252 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:38.212258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:38.212277 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:38.212556 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:38.212844 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:38.212848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:38.212850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:38.213009 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:38.213415 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:38.213423 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:38.213690 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 22:25:38.213892 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 22:25:38.214255 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":124,"threads_started":1}
I20260811 22:25:38.216490 27987 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38427
I20260811 22:25:38.216749 27987 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:38.217289 27987 heartbeater.cc:511] Master 127.24.71.254:38427 requested a full tablet report, sending...
I20260811 22:25:38.218741 27622 ts_manager.cc:194] Registered new tserver with Master: 61dc4db0d7a54008a86d4ae7a48f5bba (127.24.71.194:46023)
I20260811 22:25:38.219735 27622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:47123
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:25:38.244824 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38427
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:38.376905 27992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:38.377210 27992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:38.377256 27992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:38.381516 27992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:38.381603 27992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:38.381666 27992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:38.387352 27992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:38.388775 27992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:38.389855 27992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:38.400570 27999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:38.400583 28000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:38.400568 28002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:38.400717 27992 server_base.cc:1034] running on GCE node
I20260811 22:25:38.401321 27992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:38.402078 27992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:38.403275 27992 hybrid_clock.cc:648] HybridClock initialized: now 1786487138403252 us; error 36 us; skew 500 ppm
Aug 11 22:25:38 dist-test-slave-mvvj krb5kdc[27565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487138, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:38.406347 27992 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:25:38.407313 27992 webserver.cc:460] Webserver started at http://127.24.71.195:35001/ using document root <none> and password file <none>
I20260811 22:25:38.407966 27992 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:38.408082 27992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:38.408330 27992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:38.410522 27992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "57516a41506a4e498a7419c1f989a303"
format_stamp: "Formatted at 2026-08-11 22:25:38 on dist-test-slave-mvvj"
server_key: "567216f752abdd861a9dfee16f2f2e82"
server_key_iv: "88e407b2d4bba130a42612bcd29eb8f8"
server_key_version: "encryptionkey@0"
I20260811 22:25:38.411166 27992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "57516a41506a4e498a7419c1f989a303"
format_stamp: "Formatted at 2026-08-11 22:25:38 on dist-test-slave-mvvj"
server_key: "567216f752abdd861a9dfee16f2f2e82"
server_key_iv: "88e407b2d4bba130a42612bcd29eb8f8"
server_key_version: "encryptionkey@0"
I20260811 22:25:38.415241 27992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:38.418329 28009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:38.419764 27992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:38.420071 27992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "57516a41506a4e498a7419c1f989a303"
format_stamp: "Formatted at 2026-08-11 22:25:38 on dist-test-slave-mvvj"
server_key: "567216f752abdd861a9dfee16f2f2e82"
server_key_iv: "88e407b2d4bba130a42612bcd29eb8f8"
server_key_version: "encryptionkey@0"
I20260811 22:25:38.420194 27992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:38.438803 27992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:38.442200 27992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:38.442371 27992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:38.442946 27992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:38.444023 27992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:38.444074 27992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:38.444118 27992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:38.444134 27992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:38.454030 27992 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:36333
I20260811 22:25:38.454088 28132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:36333 every 8 connection(s)
I20260811 22:25:38.455041 27992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:38.464097 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 27992
I20260811 22:25:38.464264 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:38.464675 24863 external_mini_cluster.cc:1428] Setting key 7c583cdd7881f7ac30b7d4cb450504a8
Aug 11 22:25:38 dist-test-slave-mvvj krb5kdc[27565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487138, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:38.470140 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:38.457255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:54299 (local address 127.24.71.254:38427)
0811 22:25:38.457432 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:38.457435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:38.458523 (+ 1088us) server_negotiation.cc:407] Connection header received
0811 22:25:38.459459 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:38.459462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:38.459519 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:38.459603 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:38.461048 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:38.461638 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:38.462878 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:38.463122 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:38.463209 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:38.466269 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:38.466278 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:38.466280 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:38.466307 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:38.467812 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:38.468433 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:38.468437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:38.468439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:38.468532 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:38.468897 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:38.468902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:38.468903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:38.469110 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:38.469221 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:38.469814 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 22:25:38.469952 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:25:38.471063 28135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:38.457607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38427 (local address 127.24.71.195:54299)
0811 22:25:38.458334 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:38.458380 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:38.459228 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:38.459873 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:38.459881 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:38.460288 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:38.460843 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:38.460860 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:38.461780 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:38.461783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:38.462757 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:38.462768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:38.463298 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:38.463301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:38.463512 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:38.464264 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:38.464289 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:38.466092 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:38.467981 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:38.467988 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:38.468013 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:38.468326 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:38.468630 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:38.468633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:38.468635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:38.468799 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:38.469222 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:38.469232 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:38.469566 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 22:25:38.469772 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 22:25:38.470174 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":138,"threads_started":1}
I20260811 22:25:38.472419 28133 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38427
I20260811 22:25:38.472658 28133 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:38.473150 28133 heartbeater.cc:511] Master 127.24.71.254:38427 requested a full tablet report, sending...
I20260811 22:25:38.474291 27622 ts_manager.cc:194] Registered new tserver with Master: 57516a41506a4e498a7419c1f989a303 (127.24.71.195:36333)
I20260811 22:25:38.475003 27622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:54299
I20260811 22:25:38.479105 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:25:38.496654 27665 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:38.494698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.24.71.254:38427)
0811 22:25:38.494934 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:38.494941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:38.495004 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:25:38.495191 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:38.495195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:38.495250 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:38.495372 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:38.496502 (+ 1130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57334: BlockingRecv error: recv got EOF from 127.0.0.1:57334 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 22:25:38.498315 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 27692
I20260811 22:25:38.507264 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 27838
I20260811 22:25:38.518676 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 27992
I20260811 22:25:38.531924 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 27583
2026-08-11T22:25:38Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2398 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:38 dist-test-slave-mvvj krb5kdc[28145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:38 dist-test-slave-mvvj krb5kdc[28145](info): set up 2 sockets
Aug 11 22:25:38 dist-test-slave-mvvj krb5kdc[28145](info): commencing operation
krb5kdc: starting...
W20260811 22:25:39.780021 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:39 dist-test-slave-mvvj krb5kdc[28145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:39.932879 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38289
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:42343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:38289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:40.056725 28158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:40.057099 28158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:40.057183 28158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:40.061745 28158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:40.061901 28158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:40.061942 28158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:40.061997 28158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:40.062052 28158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:40.069139 28158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:42343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:38289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:40.070427 28158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:40.071611 28158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:40.081192 28166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:40.081300 28164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:40.081192 28163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:40.081598 28158 server_base.cc:1034] running on GCE node
I20260811 22:25:40.082312 28158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:40.083940 28158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:40.085150 28158 hybrid_clock.cc:648] HybridClock initialized: now 1786487140085136 us; error 37 us; skew 500 ppm
Aug 11 22:25:40 dist-test-slave-mvvj krb5kdc[28145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487140, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:40.088316 28158 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:40.089596 28158 webserver.cc:460] Webserver started at http://127.24.71.254:43455/ using document root <none> and password file <none>
I20260811 22:25:40.090250 28158 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:40.090319 28158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:40.090560 28158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:40.092998 28158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac"
format_stamp: "Formatted at 2026-08-11 22:25:40 on dist-test-slave-mvvj"
server_key: "67d863d01e05fa02b2c30cdd498dc71e"
server_key_iv: "c94cee6635a7250b93d1d73380af0591"
server_key_version: "encryptionkey@0"
I20260811 22:25:40.093650 28158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac"
format_stamp: "Formatted at 2026-08-11 22:25:40 on dist-test-slave-mvvj"
server_key: "67d863d01e05fa02b2c30cdd498dc71e"
server_key_iv: "c94cee6635a7250b93d1d73380af0591"
server_key_version: "encryptionkey@0"
I20260811 22:25:40.097992 28158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:40.101091 28174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:40.102794 28158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:25:40.102968 28158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac"
format_stamp: "Formatted at 2026-08-11 22:25:40 on dist-test-slave-mvvj"
server_key: "67d863d01e05fa02b2c30cdd498dc71e"
server_key_iv: "c94cee6635a7250b93d1d73380af0591"
server_key_version: "encryptionkey@0"
I20260811 22:25:40.103122 28158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:40.120639 28158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:40.124380 28158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:40.124610 28158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:40.132259 28158 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:38289
I20260811 22:25:40.132714 28234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:38289 every 8 connection(s)
I20260811 22:25:40.133473 28158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:40.137061 28235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:40.141819 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 28158
I20260811 22:25:40.141981 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:40.142299 24863 external_mini_cluster.cc:1428] Setting key 4df249fa342fd02898e926f763a7ed34
I20260811 22:25:40.144558 28235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac: Bootstrap starting.
I20260811 22:25:40.147516 28235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:40.148490 28235 log.cc:824] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:40.151073 28235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac: No bootstrap required, opened a new log
Aug 11 22:25:40 dist-test-slave-mvvj krb5kdc[28145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:40.156173 28235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38289 } }
I20260811 22:25:40.156535 28235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:40.156600 28235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1470f5e6ecdb4c5e8b5c5e2d2ad7faac, State: Initialized, Role: FOLLOWER
I20260811 22:25:40.157171 28235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38289 } }
I20260811 22:25:40.157339 28235 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:40.157416 28235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:40.157534 28235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:40.158872 28235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38289 } }
I20260811 22:25:40.159277 28235 leader_election.cc:302] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1470f5e6ecdb4c5e8b5c5e2d2ad7faac; no voters:
I20260811 22:25:40.159579 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:40.144054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37814 (local address 127.24.71.254:38289)
0811 22:25:40.144614 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:40.144650 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:40.144704 (+ 54us) server_negotiation.cc:407] Connection header received
0811 22:25:40.145490 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:40.145511 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:40.145885 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:40.146206 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:40.147278 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:40.148660 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:40.149815 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:40.150317 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:40.150454 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:40.153395 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:40.153403 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:40.153423 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:40.153460 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:40.155706 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:40.156270 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:40.156277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:40.156284 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:40.156429 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:40.156810 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:40.156813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:40.156815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:40.157397 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:40.157637 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:40.158015 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 22:25:40.158397 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":163,"threads_started":1}
I20260811 22:25:40.159673 28235 leader_election.cc:288] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:40.159816 28242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:40.160123 28242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [term 1 LEADER]: Becoming Leader. State: Replica: 1470f5e6ecdb4c5e8b5c5e2d2ad7faac, State: Running, Role: LEADER
I20260811 22:25:40.160542 28242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38289 } }
I20260811 22:25:40.161082 28235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:40.162341 28244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38289 } } }
I20260811 22:25:40.162498 28244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:40.162945 28252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:40.162896 28243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1470f5e6ecdb4c5e8b5c5e2d2ad7faac. Latest consensus state: current_term: 1 leader_uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1470f5e6ecdb4c5e8b5c5e2d2ad7faac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38289 } } }
I20260811 22:25:40.163002 28243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:40.163220 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:40.166671 28252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:40.176486 28252 catalog_manager.cc:1347] Generated new cluster ID: 3926b5ecd0414c8ea1302b63b34cfde2
I20260811 22:25:40.176658 28252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:40.187304 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:40.181797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37818 (local address 127.24.71.254:38289)
0811 22:25:40.181947 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:40.181952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:40.182237 (+ 285us) server_negotiation.cc:407] Connection header received
0811 22:25:40.182423 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:40.182426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:40.182477 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:40.182578 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:40.183235 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:40.183923 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:40.184847 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:40.185108 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:40.185223 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:40.186203 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:40.186207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:40.186210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:40.186770 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:40.186807 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:40.186825 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:40.186935 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:40.187054 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 22:25:40.187135 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:25:40.196843 28252 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:40.198168 28252 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:40.204008 28252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1470f5e6ecdb4c5e8b5c5e2d2ad7faac: Generated new TSK 0
I20260811 22:25:40.204957 28252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:40.211124 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 28158
2026-08-11T22:25:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1679 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:40 dist-test-slave-mvvj krb5kdc[28271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:40 dist-test-slave-mvvj krb5kdc[28271](info): set up 2 sockets
Aug 11 22:25:40 dist-test-slave-mvvj krb5kdc[28271](info): commencing operation
krb5kdc: starting...
W20260811 22:25:41.366107 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:41 dist-test-slave-mvvj krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:41.512347 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:39147
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:39733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:39147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:41.635218 28285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:41.635509 28285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:41.635555 28285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:41.639654 28285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:41.639724 28285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:41.639745 28285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:41.639762 28285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:41.639777 28285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:41.645211 28285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:39733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:39147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:39147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:41.646337 28285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:41.647367 28285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:41.656985 28290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:41.657061 28285 server_base.cc:1034] running on GCE node
W20260811 22:25:41.656994 28291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:41.656996 28293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:41.657935 28285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:41.659251 28285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:41.660456 28285 hybrid_clock.cc:648] HybridClock initialized: now 1786487141660419 us; error 52 us; skew 500 ppm
Aug 11 22:25:41 dist-test-slave-mvvj krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487141, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:41.669086 28285 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:41.670173 28285 webserver.cc:460] Webserver started at http://127.24.71.254:45737/ using document root <none> and password file <none>
I20260811 22:25:41.670821 28285 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:41.670893 28285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:41.671085 28285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:41.673522 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "9426c00a1dc5449fa417a2873f2e1cb6"
format_stamp: "Formatted at 2026-08-11 22:25:41 on dist-test-slave-mvvj"
server_key: "998571b223c2949b0c3f750334f24088"
server_key_iv: "44b2827447c3247aa3ff1bf25a1e1976"
server_key_version: "encryptionkey@0"
I20260811 22:25:41.674139 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "9426c00a1dc5449fa417a2873f2e1cb6"
format_stamp: "Formatted at 2026-08-11 22:25:41 on dist-test-slave-mvvj"
server_key: "998571b223c2949b0c3f750334f24088"
server_key_iv: "44b2827447c3247aa3ff1bf25a1e1976"
server_key_version: "encryptionkey@0"
I20260811 22:25:41.678668 28285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 22:25:41.681628 28300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:41.683051 28285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:41.683229 28285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "9426c00a1dc5449fa417a2873f2e1cb6"
format_stamp: "Formatted at 2026-08-11 22:25:41 on dist-test-slave-mvvj"
server_key: "998571b223c2949b0c3f750334f24088"
server_key_iv: "44b2827447c3247aa3ff1bf25a1e1976"
server_key_version: "encryptionkey@0"
I20260811 22:25:41.683368 28285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:41.694751 28285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:41.698534 28285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:41.698772 28285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:41.706525 28285 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:39147
I20260811 22:25:41.706815 28357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:39147 every 8 connection(s)
I20260811 22:25:41.707588 28285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:41.710590 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 28285
I20260811 22:25:41.710731 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:41.710785 28358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:41.711069 24863 external_mini_cluster.cc:1428] Setting key b3af5b9809e8beb126155f291ed86aa2
I20260811 22:25:41.718766 28358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6: Bootstrap starting.
Aug 11 22:25:41 dist-test-slave-mvvj krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:41.721727 28358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:41.722674 28358 log.cc:824] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:41.724892 28358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6: No bootstrap required, opened a new log
I20260811 22:25:41.726770 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:41.712617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.24.71.254:39147)
0811 22:25:41.713063 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:41.713079 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:41.713120 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:25:41.713817 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:41.713837 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:41.714196 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:41.714499 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:41.715478 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:41.716568 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:41.717760 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:41.718239 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:41.718379 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:41.721384 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:41.721391 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:41.721415 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:41.721450 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:41.723439 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:41.724010 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:41.724016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:41.724021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:41.724144 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:41.724465 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:41.724468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:41.724469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:41.724894 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:41.725114 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:41.725425 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 22:25:41.725741 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":143,"threads_started":1}
I20260811 22:25:41.728425 28358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426c00a1dc5449fa417a2873f2e1cb6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39147 } }
I20260811 22:25:41.728739 28358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:41.728792 28358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9426c00a1dc5449fa417a2873f2e1cb6, State: Initialized, Role: FOLLOWER
I20260811 22:25:41.729287 28358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426c00a1dc5449fa417a2873f2e1cb6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39147 } }
I20260811 22:25:41.729465 28358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:41.729539 28358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:41.729619 28358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:41.730634 28358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426c00a1dc5449fa417a2873f2e1cb6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39147 } }
I20260811 22:25:41.730958 28358 leader_election.cc:302] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9426c00a1dc5449fa417a2873f2e1cb6; no voters:
I20260811 22:25:41.731251 28358 leader_election.cc:288] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:41.731413 28363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:41.731659 28363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [term 1 LEADER]: Becoming Leader. State: Replica: 9426c00a1dc5449fa417a2873f2e1cb6, State: Running, Role: LEADER
I20260811 22:25:41.732074 28363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426c00a1dc5449fa417a2873f2e1cb6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39147 } }
I20260811 22:25:41.732678 28358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:41.734054 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9426c00a1dc5449fa417a2873f2e1cb6. Latest consensus state: current_term: 1 leader_uuid: "9426c00a1dc5449fa417a2873f2e1cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426c00a1dc5449fa417a2873f2e1cb6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39147 } } }
I20260811 22:25:41.734054 28365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9426c00a1dc5449fa417a2873f2e1cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426c00a1dc5449fa417a2873f2e1cb6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39147 } } }
I20260811 22:25:41.734220 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:41.734220 28365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:41.734671 28374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:41.738252 28374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:41.744858 28374 catalog_manager.cc:1347] Generated new cluster ID: a4f2d1c542cf4db09f16c0e722829e2a
I20260811 22:25:41.744995 28374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:41.760789 28374 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:41.761904 28374 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:41.767385 28374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9426c00a1dc5449fa417a2873f2e1cb6: Generated new TSK 0
I20260811 22:25:41.768299 28374 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:25:41.808072 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39147
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:39733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:41.948349 28389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:41.948760 28389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:41.948865 28389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:41.953234 28389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:41.953328 28389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:41.953421 28389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:41.959152 28389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:39733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:41.960481 28389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:41.961632 28389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:41.972524 28389 server_base.cc:1034] running on GCE node
W20260811 22:25:41.972581 28395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:41.972576 28397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:41.972812 28394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:41.973256 28389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:41.974035 28389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:41.975274 28389 hybrid_clock.cc:648] HybridClock initialized: now 1786487141975232 us; error 51 us; skew 500 ppm
Aug 11 22:25:41 dist-test-slave-mvvj krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487141, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:41.978737 28389 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:25:41.979976 28389 webserver.cc:460] Webserver started at http://127.24.71.193:36361/ using document root <none> and password file <none>
I20260811 22:25:41.980652 28389 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:41.980756 28389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:41.981021 28389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:41.983497 28389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "1b7ec82171c24a158df2fe176115ec20"
format_stamp: "Formatted at 2026-08-11 22:25:41 on dist-test-slave-mvvj"
server_key: "86744e5cb976131c6375aeffc9375872"
server_key_iv: "da45e0fa73d8e9b18b977fced9aac9c5"
server_key_version: "encryptionkey@0"
I20260811 22:25:41.984216 28389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "1b7ec82171c24a158df2fe176115ec20"
format_stamp: "Formatted at 2026-08-11 22:25:41 on dist-test-slave-mvvj"
server_key: "86744e5cb976131c6375aeffc9375872"
server_key_iv: "da45e0fa73d8e9b18b977fced9aac9c5"
server_key_version: "encryptionkey@0"
I20260811 22:25:41.988809 28389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:25:41.991652 28405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:41.993088 28389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:25:41.993251 28389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "1b7ec82171c24a158df2fe176115ec20"
format_stamp: "Formatted at 2026-08-11 22:25:41 on dist-test-slave-mvvj"
server_key: "86744e5cb976131c6375aeffc9375872"
server_key_iv: "da45e0fa73d8e9b18b977fced9aac9c5"
server_key_version: "encryptionkey@0"
I20260811 22:25:41.993391 28389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:42.008638 28389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:42.012342 28389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:42.012562 28389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:42.013312 28389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:42.014245 28389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:42.014307 28389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:42.014351 28389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:42.014367 28389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:42.023779 28389 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:37553
I20260811 22:25:42.023903 28534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:37553 every 8 connection(s)
I20260811 22:25:42.024837 28389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:42.027330 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 28389
I20260811 22:25:42.027477 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:42.027868 24863 external_mini_cluster.cc:1428] Setting key ac5e6476935c3936495f84d5e31d7258
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487141, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:25:42.042173 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:42.026931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:46355 (local address 127.24.71.254:39147)
0811 22:25:42.027077 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:42.027080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:42.028256 (+ 1176us) server_negotiation.cc:407] Connection header received
0811 22:25:42.029973 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:42.029976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:42.030024 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:42.030096 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:42.031809 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.032790 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.034417 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.034898 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.035040 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:42.037841 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:42.037848 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:42.037851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:42.037889 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:42.039695 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:42.040471 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:42.040475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:42.040476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:42.040570 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:42.040923 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:42.040925 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:42.040927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:42.041129 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:42.041218 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:42.041896 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 22:25:42.042033 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
I20260811 22:25:42.043292 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:42.027408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:39147 (local address 127.24.71.193:46355)
0811 22:25:42.028090 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:42.028141 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:42.029721 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:42.030284 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:42.030298 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:42.030854 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:42.031608 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:42.031636 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.032962 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.032965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:42.034234 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:42.034246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.035092 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.035095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:42.035256 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:42.035971 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:42.035990 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:42.037602 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:42.039936 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:42.039942 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:42.039959 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:42.040305 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:42.040667 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:42.040669 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:42.040670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:42.040824 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:42.041227 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:42.041234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:42.041613 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 22:25:42.041796 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 22:25:42.042098 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":157,"threads_started":1}
I20260811 22:25:42.044914 28535 heartbeater.cc:347] Connected to a master server at 127.24.71.254:39147
I20260811 22:25:42.045207 28535 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:42.045768 28535 heartbeater.cc:511] Master 127.24.71.254:39147 requested a full tablet report, sending...
I20260811 22:25:42.047709 28319 ts_manager.cc:194] Registered new tserver with Master: 1b7ec82171c24a158df2fe176115ec20 (127.24.71.193:37553)
I20260811 22:25:42.049666 28319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:46355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:25:42.064023 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39147
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:39733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:42.185150 28541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:42.185464 28541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:42.185510 28541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:42.189863 28541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:42.189949 28541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:42.190040 28541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:42.195653 28541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:39733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:42.196977 28541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:42.198143 28541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:42.208523 28546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:42.208789 28541 server_base.cc:1034] running on GCE node
W20260811 22:25:42.208521 28547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:42.208523 28549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:42.209393 28541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:42.210184 28541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:42.211491 28541 hybrid_clock.cc:648] HybridClock initialized: now 1786487142211473 us; error 30 us; skew 500 ppm
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:42.214704 28541 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:25:42.215791 28541 webserver.cc:460] Webserver started at http://127.24.71.194:39081/ using document root <none> and password file <none>
I20260811 22:25:42.216487 28541 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:42.216585 28541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:42.216832 28541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:42.219404 28541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "2f78dd65bf9b401aa1ff713523082053"
format_stamp: "Formatted at 2026-08-11 22:25:42 on dist-test-slave-mvvj"
server_key: "de26213fdea268bd3b177a7ec24268c8"
server_key_iv: "8192bf310892e4b20b27fa4dbc365202"
server_key_version: "encryptionkey@0"
I20260811 22:25:42.220217 28541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "2f78dd65bf9b401aa1ff713523082053"
format_stamp: "Formatted at 2026-08-11 22:25:42 on dist-test-slave-mvvj"
server_key: "de26213fdea268bd3b177a7ec24268c8"
server_key_iv: "8192bf310892e4b20b27fa4dbc365202"
server_key_version: "encryptionkey@0"
I20260811 22:25:42.224961 28541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:25:42.228137 28556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:42.229799 28541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:25:42.229987 28541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "2f78dd65bf9b401aa1ff713523082053"
format_stamp: "Formatted at 2026-08-11 22:25:42 on dist-test-slave-mvvj"
server_key: "de26213fdea268bd3b177a7ec24268c8"
server_key_iv: "8192bf310892e4b20b27fa4dbc365202"
server_key_version: "encryptionkey@0"
I20260811 22:25:42.230129 28541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:42.242121 28541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:42.245581 28541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:42.245812 28541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:42.246444 28541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:42.247385 28541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:42.247434 28541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:42.247505 28541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:42.247540 28541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:42.256815 28541 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:44483
I20260811 22:25:42.256896 28680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:44483 every 8 connection(s)
I20260811 22:25:42.257874 28541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:42.263008 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 28541
I20260811 22:25:42.263157 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:42.263514 24863 external_mini_cluster.cc:1428] Setting key f40c0b15f4884297113d5054e86842e2
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:42.274756 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:42.260132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:51521 (local address 127.24.71.254:39147)
0811 22:25:42.260314 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:42.260319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:42.261319 (+ 1000us) server_negotiation.cc:407] Connection header received
0811 22:25:42.262816 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:42.262823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:42.262896 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:42.263028 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:42.264627 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.265261 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.266431 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.266666 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.266778 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:42.270116 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:42.270120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:42.270123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:42.270152 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:42.271995 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:42.272742 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:42.272745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:42.272747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:42.272838 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:42.273276 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:42.273279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:42.273280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:42.273558 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:42.273649 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:42.274466 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 22:25:42.274617 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:25:42.276016 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:42.260633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:39147 (local address 127.24.71.194:51521)
0811 22:25:42.261155 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:42.261201 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:42.262544 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:42.263301 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:42.263311 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:42.263820 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:42.264473 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:42.264488 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.265394 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.265397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:42.266295 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:42.266305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.266827 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.266832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:42.267004 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:42.267689 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:42.267708 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:42.269897 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:42.272135 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:42.272144 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:42.272166 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:42.272584 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:42.272944 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:42.272950 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:42.272954 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:42.273166 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:42.273674 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:42.273684 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:42.274074 (+ 390us) client_negotiation.cc:769] Sending connection context
0811 22:25:42.274313 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 22:25:42.274715 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":131,"threads_started":1}
I20260811 22:25:42.277468 28681 heartbeater.cc:347] Connected to a master server at 127.24.71.254:39147
I20260811 22:25:42.277763 28681 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:42.278424 28681 heartbeater.cc:511] Master 127.24.71.254:39147 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:25:42.279695 28319 ts_manager.cc:194] Registered new tserver with Master: 2f78dd65bf9b401aa1ff713523082053 (127.24.71.194:44483)
I20260811 22:25:42.280414 28319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:51521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:25:42.299496 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39147
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:39733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:42.423415 28687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:42.423753 28687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:42.423823 28687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:42.427995 28687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:42.428085 28687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:42.428174 28687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:42.433907 28687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:39733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:42.435153 28687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:42.436290 28687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:42.446964 28695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:42.446969 28692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:42.447100 28687 server_base.cc:1034] running on GCE node
W20260811 22:25:42.446957 28693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:42.447806 28687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:42.448551 28687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:42.449766 28687 hybrid_clock.cc:648] HybridClock initialized: now 1786487142449728 us; error 57 us; skew 500 ppm
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:42.452898 28687 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:25:42.454069 28687 webserver.cc:460] Webserver started at http://127.24.71.195:46443/ using document root <none> and password file <none>
I20260811 22:25:42.454722 28687 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:42.454820 28687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:42.455070 28687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:42.457552 28687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "7ed3ce7ab1684280913fc03073a93212"
format_stamp: "Formatted at 2026-08-11 22:25:42 on dist-test-slave-mvvj"
server_key: "cab9ad231ce1f874e74597cfc4b6a98e"
server_key_iv: "1dde56e92c414aa3fffbef5fc783bceb"
server_key_version: "encryptionkey@0"
I20260811 22:25:42.458212 28687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "7ed3ce7ab1684280913fc03073a93212"
format_stamp: "Formatted at 2026-08-11 22:25:42 on dist-test-slave-mvvj"
server_key: "cab9ad231ce1f874e74597cfc4b6a98e"
server_key_iv: "1dde56e92c414aa3fffbef5fc783bceb"
server_key_version: "encryptionkey@0"
I20260811 22:25:42.462669 28687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:42.465469 28703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:42.466830 28687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:42.466975 28687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "7ed3ce7ab1684280913fc03073a93212"
format_stamp: "Formatted at 2026-08-11 22:25:42 on dist-test-slave-mvvj"
server_key: "cab9ad231ce1f874e74597cfc4b6a98e"
server_key_iv: "1dde56e92c414aa3fffbef5fc783bceb"
server_key_version: "encryptionkey@0"
I20260811 22:25:42.467103 28687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:42.491947 28687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:42.495499 28687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:42.495725 28687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:42.496421 28687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:42.497635 28687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:42.497723 28687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:42.497802 28687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:42.497828 28687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:42.508155 28687 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:34881
I20260811 22:25:42.508200 28829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:34881 every 8 connection(s)
I20260811 22:25:42.509137 28687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:42.510001 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 28687
I20260811 22:25:42.510149 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:42.510460 24863 external_mini_cluster.cc:1428] Setting key e093870936cbd25ecd6fbde5ee9c83a4
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487142, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:42.524302 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:42.511199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:40771 (local address 127.24.71.254:39147)
0811 22:25:42.511369 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:42.511374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:42.512320 (+ 946us) server_negotiation.cc:407] Connection header received
0811 22:25:42.513967 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:42.513970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:42.514017 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:42.514103 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:42.515503 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.516094 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.517321 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.517590 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.517681 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:42.520473 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:42.520476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:42.520478 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:42.520507 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:42.521961 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:42.522507 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:42.522510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:42.522511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:42.522594 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:42.522969 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:42.522972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:42.522973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:42.523173 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:42.523289 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:42.523997 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 22:25:42.524134 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:25:42.525379 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:42.511546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:39147 (local address 127.24.71.195:40771)
0811 22:25:42.512154 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:42.512219 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:42.513720 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:42.514333 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:42.514343 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:42.514755 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:42.515335 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:42.515351 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.516222 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.516225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:42.517188 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:42.517199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.517741 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.517744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:42.517939 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:42.518622 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:42.518639 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:42.520285 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:42.522089 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:42.522094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:42.522108 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:42.522404 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:42.522702 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:42.522706 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:42.522707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:42.522857 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:42.523336 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:42.523343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:42.523677 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 22:25:42.523936 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 22:25:42.524312 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":184,"threads_started":1}
I20260811 22:25:42.526943 28830 heartbeater.cc:347] Connected to a master server at 127.24.71.254:39147
I20260811 22:25:42.527215 28830 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:42.527858 28830 heartbeater.cc:511] Master 127.24.71.254:39147 requested a full tablet report, sending...
I20260811 22:25:42.529253 28319 ts_manager.cc:194] Registered new tserver with Master: 7ed3ce7ab1684280913fc03073a93212 (127.24.71.195:34881)
I20260811 22:25:42.530000 28319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:40771
I20260811 22:25:42.537281 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487142, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487142, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:42.567934 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:42.557582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.24.71.254:39147)
0811 22:25:42.557743 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:42.557747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:42.557881 (+ 134us) server_negotiation.cc:407] Connection header received
0811 22:25:42.557975 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:42.557978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:42.558029 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:42.558118 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:42.559013 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.559598 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.560557 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:42.560782 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:42.560891 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:42.563704 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:42.563707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:42.563710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:42.563742 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:42.565632 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:42.566255 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:42.566259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:42.566260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:42.566378 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:42.566786 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:42.566789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:42.566790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:42.567074 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:42.567205 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:42.567582 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 22:25:42.567724 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 22:25:42.568408 28319 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53876
I20260811 22:25:42.569455 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 28389
I20260811 22:25:42.579052 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 28541
I20260811 22:25:42.587477 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 28687
I20260811 22:25:42.595269 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 28285
2026-08-11T22:25:42Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2385 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28844](info): set up 2 sockets
Aug 11 22:25:42 dist-test-slave-mvvj krb5kdc[28844](info): commencing operation
krb5kdc: starting...
W20260811 22:25:43.718362 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:43 dist-test-slave-mvvj krb5kdc[28844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:43.863029 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38845
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:38845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:43.983456 28859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:43.984124 28859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:43.984246 28859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:43.988812 28859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:43.988950 28859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:43.988986 28859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:43.989045 28859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:43.989101 28859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:43.995018 28859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:38845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:43.996407 28859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:43.997571 28859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:44.007368 28868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:44.007454 28864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:44.007386 28865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:44.007699 28859 server_base.cc:1034] running on GCE node
I20260811 22:25:44.008471 28859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:44.009761 28859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:44.010973 28859 hybrid_clock.cc:648] HybridClock initialized: now 1786487144010944 us; error 58 us; skew 500 ppm
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:44.014057 28859 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:44.015075 28859 webserver.cc:460] Webserver started at http://127.24.71.254:38197/ using document root <none> and password file <none>
I20260811 22:25:44.015687 28859 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:44.015785 28859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:44.016060 28859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:44.018394 28859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "c9e36e49f1104b01ac4af59068eb321c"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "ed69c6e4b4d6b7d67023f781a4973282"
server_key_iv: "abd101b1cc63de4274aa829533578c9a"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.019070 28859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "c9e36e49f1104b01ac4af59068eb321c"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "ed69c6e4b4d6b7d67023f781a4973282"
server_key_iv: "abd101b1cc63de4274aa829533578c9a"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.023432 28859 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 22:25:44.026213 28876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.027655 28859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:44.027823 28859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "c9e36e49f1104b01ac4af59068eb321c"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "ed69c6e4b4d6b7d67023f781a4973282"
server_key_iv: "abd101b1cc63de4274aa829533578c9a"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.028012 28859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:44.038319 28859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:44.041550 28859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:44.041765 28859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:44.049773 28859 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:38845
I20260811 22:25:44.049805 28933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:38845 every 8 connection(s)
I20260811 22:25:44.050858 28859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:44.051106 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 28859
I20260811 22:25:44.051250 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:44.051623 24863 external_mini_cluster.cc:1428] Setting key c743ecce9efc9dfc5a09ddab8ebd18a8
I20260811 22:25:44.054461 28934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:44.061333 28934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c: Bootstrap starting.
I20260811 22:25:44.064090 28934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:44.064931 28934 log.cc:824] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:44.066905 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:44.053174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55886 (local address 127.24.71.254:38845)
0811 22:25:44.053725 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:44.053742 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:44.053782 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:25:44.054497 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:44.054516 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:44.054882 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:44.055273 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:44.056201 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.057391 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.058260 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.058772 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.058949 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:44.061938 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:44.061943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:44.061959 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:44.061992 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:44.063786 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.064355 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.064362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.064367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.064492 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.064802 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.064806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.064808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.065297 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:44.065528 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:44.065811 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:25:44.066131 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":162,"threads_started":1}
I20260811 22:25:44.067194 28934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c: No bootstrap required, opened a new log
I20260811 22:25:44.071460 28934 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e36e49f1104b01ac4af59068eb321c" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38845 } }
I20260811 22:25:44.071744 28934 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:44.071857 28934 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9e36e49f1104b01ac4af59068eb321c, State: Initialized, Role: FOLLOWER
I20260811 22:25:44.072315 28934 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e36e49f1104b01ac4af59068eb321c" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38845 } }
I20260811 22:25:44.072474 28934 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:44.072561 28934 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:44.072685 28934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:44.073779 28934 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e36e49f1104b01ac4af59068eb321c" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38845 } }
I20260811 22:25:44.074213 28934 leader_election.cc:302] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9e36e49f1104b01ac4af59068eb321c; no voters:
I20260811 22:25:44.074615 28934 leader_election.cc:288] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:44.074748 28939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:44.075088 28939 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [term 1 LEADER]: Becoming Leader. State: Replica: c9e36e49f1104b01ac4af59068eb321c, State: Running, Role: LEADER
I20260811 22:25:44.075443 28939 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e36e49f1104b01ac4af59068eb321c" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38845 } }
I20260811 22:25:44.076078 28934 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:44.077363 28941 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9e36e49f1104b01ac4af59068eb321c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e36e49f1104b01ac4af59068eb321c" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38845 } } }
I20260811 22:25:44.077512 28941 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:44.077740 28940 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9e36e49f1104b01ac4af59068eb321c. Latest consensus state: current_term: 1 leader_uuid: "c9e36e49f1104b01ac4af59068eb321c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e36e49f1104b01ac4af59068eb321c" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38845 } } }
I20260811 22:25:44.077924 28940 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:44.077991 28949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:44.081871 28949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:44.088824 28949 catalog_manager.cc:1347] Generated new cluster ID: 365a4c7e99fa4d7da8a6556aa0e6c19e
I20260811 22:25:44.088956 28949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:44.106135 28949 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:44.107211 28949 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:44.116462 28949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9e36e49f1104b01ac4af59068eb321c: Generated new TSK 0
I20260811 22:25:44.117259 28949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:25:44.152486 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38845
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:44.284791 28963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:44.285084 28963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:44.285130 28963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:44.289285 28963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:44.289363 28963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:44.289424 28963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:44.294952 28963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:44.296136 28963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:44.297232 28963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:44.307809 28969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:44.307888 28963 server_base.cc:1034] running on GCE node
W20260811 22:25:44.307814 28968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:44.307809 28971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:44.308671 28963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:44.309362 28963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:44.310570 28963 hybrid_clock.cc:648] HybridClock initialized: now 1786487144310553 us; error 29 us; skew 500 ppm
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:44.313702 28963 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:25:44.314649 28963 webserver.cc:460] Webserver started at http://127.24.71.193:44083/ using document root <none> and password file <none>
I20260811 22:25:44.315222 28963 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:44.315284 28963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:44.315464 28963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:44.317755 28963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "1c4be447df7f4608998862de0b2a82b3"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "d4923e07680e715e6822bd6609889450"
server_key_iv: "de3b97b446f2f416cb219a002f1ee8e8"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.318352 28963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "1c4be447df7f4608998862de0b2a82b3"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "d4923e07680e715e6822bd6609889450"
server_key_iv: "de3b97b446f2f416cb219a002f1ee8e8"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.322511 28963 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:25:44.325515 28978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.326884 28963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:44.327015 28963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "1c4be447df7f4608998862de0b2a82b3"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "d4923e07680e715e6822bd6609889450"
server_key_iv: "de3b97b446f2f416cb219a002f1ee8e8"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.327117 28963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:44.346661 28963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:44.350167 28963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:44.350335 28963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:44.350905 28963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:44.351771 28963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:44.351820 28963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.351897 28963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:44.351917 28963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.361084 28963 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:38683
I20260811 22:25:44.361120 29105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:38683 every 8 connection(s)
I20260811 22:25:44.362048 28963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:44.371331 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 28963
I20260811 22:25:44.371433 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:44.371795 24863 external_mini_cluster.cc:1428] Setting key feb8142d42245b744208974c23a2be7a
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:44.377801 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:44.364077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:43635 (local address 127.24.71.254:38845)
0811 22:25:44.364226 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:44.364230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:44.365094 (+ 864us) server_negotiation.cc:407] Connection header received
0811 22:25:44.366595 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:44.366599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:44.366662 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:44.366766 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:44.368003 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.368538 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.369733 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.369971 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.370080 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:44.372843 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:44.372847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:44.372849 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:44.372874 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:44.374742 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.375551 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.375556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.375558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.375696 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.376158 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.376164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.376167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.376475 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:44.376599 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:44.377336 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 22:25:44.377522 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:25:44.378646 29108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:44.364460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38845 (local address 127.24.71.193:43635)
0811 22:25:44.364932 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:44.364977 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:44.366318 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:44.366954 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:44.366962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:44.367332 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:44.367820 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:44.367852 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.368683 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.368686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:44.369600 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:44.369609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.370131 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.370136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:44.370300 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:44.370893 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:44.370909 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:44.372672 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:44.374886 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:44.374897 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:44.374921 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:44.375376 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:44.375824 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:44.375826 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:44.375828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:44.376040 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:44.376592 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:44.376598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:44.377041 (+ 443us) client_negotiation.cc:769] Sending connection context
0811 22:25:44.377219 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 22:25:44.377562 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260811 22:25:44.380069 29106 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38845
I20260811 22:25:44.380335 29106 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:44.380870 29106 heartbeater.cc:511] Master 127.24.71.254:38845 requested a full tablet report, sending...
I20260811 22:25:44.382753 28895 ts_manager.cc:194] Registered new tserver with Master: 1c4be447df7f4608998862de0b2a82b3 (127.24.71.193:38683)
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:25:44.384609 28895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:43635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:25:44.407117 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38845
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:44.527000 29112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:44.527281 29112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:44.527366 29112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:44.532251 29112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:44.532335 29112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:44.532419 29112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:44.539415 29112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:44.540670 29112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:44.541777 29112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:44.551941 29120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:44.552012 29117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:44.551941 29118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:44.552261 29112 server_base.cc:1034] running on GCE node
I20260811 22:25:44.552663 29112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:44.553330 29112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:44.554560 29112 hybrid_clock.cc:648] HybridClock initialized: now 1786487144554544 us; error 32 us; skew 500 ppm
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:44.557612 29112 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:25:44.558642 29112 webserver.cc:460] Webserver started at http://127.24.71.194:39965/ using document root <none> and password file <none>
I20260811 22:25:44.559234 29112 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:44.559326 29112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:44.559571 29112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:44.561784 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "e5e47ca0ef134da282ddb4ac09b7e592"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "f33a36a01b4c3e13773627b502fa37e3"
server_key_iv: "6e31c4c05b3a5a8b75670a9cc1c6bae9"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.562388 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "e5e47ca0ef134da282ddb4ac09b7e592"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "f33a36a01b4c3e13773627b502fa37e3"
server_key_iv: "6e31c4c05b3a5a8b75670a9cc1c6bae9"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.566493 29112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:44.569208 29128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.570509 29112 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:25:44.570648 29112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "e5e47ca0ef134da282ddb4ac09b7e592"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "f33a36a01b4c3e13773627b502fa37e3"
server_key_iv: "6e31c4c05b3a5a8b75670a9cc1c6bae9"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.570772 29112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:44.585319 29112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:44.588953 29112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:44.589184 29112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:44.589848 29112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:44.590777 29112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:44.590850 29112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.590921 29112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:44.590968 29112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.600395 29112 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:40849
I20260811 22:25:44.600416 29254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:40849 every 8 connection(s)
I20260811 22:25:44.601413 29112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:44.605472 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 29112
I20260811 22:25:44.605598 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:44.605978 24863 external_mini_cluster.cc:1428] Setting key d9101c8a316614395d1c0d9f28d01dc9
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:44.617053 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:44.603359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:50137 (local address 127.24.71.254:38845)
0811 22:25:44.603579 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:44.603583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:44.604348 (+ 765us) server_negotiation.cc:407] Connection header received
0811 22:25:44.605830 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:44.605834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:44.605900 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:44.606004 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:44.607469 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.608251 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.609497 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.609797 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.609882 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:44.612707 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:44.612714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:44.612720 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:44.612755 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:44.614638 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.615163 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.615170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.615174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.615283 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.615642 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.615648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.615651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.616011 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:44.616124 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:44.616741 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 22:25:44.616875 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:25:44.617908 29257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:44.603665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38845 (local address 127.24.71.194:50137)
0811 22:25:44.604190 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:44.604235 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:44.605563 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:44.606254 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:44.606262 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:44.606757 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:44.607275 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:44.607313 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.608397 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.608401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:44.609311 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:44.609322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.609960 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.609967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:44.610224 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:44.610993 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:44.611017 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:44.612530 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:44.614768 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:44.614773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:44.614786 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:44.615056 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:44.615383 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:44.615385 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:44.615387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:44.615524 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:44.616158 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:44.616166 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:44.616449 (+ 283us) client_negotiation.cc:769] Sending connection context
0811 22:25:44.616690 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 22:25:44.617040 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":184,"threads_started":1}
I20260811 22:25:44.619210 29255 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38845
I20260811 22:25:44.619437 29255 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:44.619987 29255 heartbeater.cc:511] Master 127.24.71.254:38845 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:25:44.621140 28895 ts_manager.cc:194] Registered new tserver with Master: e5e47ca0ef134da282ddb4ac09b7e592 (127.24.71.194:40849)
I20260811 22:25:44.622008 28895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:50137
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:25:44.639613 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38845
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:44.758061 29263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:44.758363 29263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:44.758409 29263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:44.762739 29263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:44.762820 29263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:44.762917 29263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:44.768568 29263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:44.769737 29263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:44.770794 29263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:44.781803 29268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:44.781924 29271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:44.781994 29269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:44.782258 29263 server_base.cc:1034] running on GCE node
I20260811 22:25:44.782732 29263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:44.783486 29263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:44.784693 29263 hybrid_clock.cc:648] HybridClock initialized: now 1786487144784653 us; error 49 us; skew 500 ppm
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:44.787932 29263 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:25:44.788904 29263 webserver.cc:460] Webserver started at http://127.24.71.195:41517/ using document root <none> and password file <none>
I20260811 22:25:44.789491 29263 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:44.789580 29263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:44.789813 29263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:44.791990 29263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "0e53007028ad467e90574592166839a1"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "a705930d6443ca2c9755f50307b12243"
server_key_iv: "a11acd8efbda34554d392f7fad11c4cf"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.792552 29263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "0e53007028ad467e90574592166839a1"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "a705930d6443ca2c9755f50307b12243"
server_key_iv: "a11acd8efbda34554d392f7fad11c4cf"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.796535 29263 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:25:44.799429 29278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.800843 29263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:44.801011 29263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "0e53007028ad467e90574592166839a1"
format_stamp: "Formatted at 2026-08-11 22:25:44 on dist-test-slave-mvvj"
server_key: "a705930d6443ca2c9755f50307b12243"
server_key_iv: "a11acd8efbda34554d392f7fad11c4cf"
server_key_version: "encryptionkey@0"
I20260811 22:25:44.801153 29263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:44.834825 29263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:44.838693 29263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:44.838936 29263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:44.840101 29263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:44.841101 29263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:44.841179 29263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.841244 29263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:44.841284 29263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:44.851052 29263 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:41407
I20260811 22:25:44.851073 29406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:41407 every 8 connection(s)
I20260811 22:25:44.852146 29263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:44.858091 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 29263
I20260811 22:25:44.858255 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:44.858620 24863 external_mini_cluster.cc:1428] Setting key 8d2fb9274e69e006bd7fdf292d9b0869
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:44.868752 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:44.854280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:44205 (local address 127.24.71.254:38845)
0811 22:25:44.854513 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:44.854518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:44.855420 (+ 902us) server_negotiation.cc:407] Connection header received
0811 22:25:44.856959 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:44.856962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:44.857074 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:44.857171 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:44.858978 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.859738 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.861325 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.861581 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.861673 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:44.864626 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:44.864630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:44.864632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:44.864664 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:44.866290 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.866989 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.866993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.866994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.867091 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.867529 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.867532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.867533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.867746 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:44.867896 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:44.868480 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 22:25:44.868606 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 22:25:44.869757 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:44.854709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38845 (local address 127.24.71.195:44205)
0811 22:25:44.855234 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:44.855289 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:44.856707 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:44.857372 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:44.857380 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:44.857925 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:44.858704 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:44.858744 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.859922 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.859926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:44.861202 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:44.861212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.861755 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.861758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:44.861927 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:44.862576 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:44.862594 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:44.864407 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:44.866453 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:44.866460 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:44.866476 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:44.866835 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:44.867215 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:44.867218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:44.867220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:44.867409 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:44.867891 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:44.867900 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:44.868208 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 22:25:44.868391 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 22:25:44.868708 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":144,"threads_started":1}
I20260811 22:25:44.871206 29407 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38845
I20260811 22:25:44.871454 29407 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:44.872161 29407 heartbeater.cc:511] Master 127.24.71.254:38845 requested a full tablet report, sending...
I20260811 22:25:44.873517 28894 ts_manager.cc:194] Registered new tserver with Master: 0e53007028ad467e90574592166839a1 (127.24.71.195:41407)
I20260811 22:25:44.874253 28894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:44205
I20260811 22:25:44.886413 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:25:44.915526 29415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:44.906087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.24.71.193:38683)
0811 22:25:44.906415 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:44.906421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:44.906449 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:25:44.906513 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:44.906518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:44.906713 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:44.906837 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:44.907582 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.908407 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.909221 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.909620 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.909705 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:44.912232 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:44.912242 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:44.912255 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:44.912281 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:44.913805 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.914227 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.914232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.914233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.914312 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.914549 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.914551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.914552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.914789 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:44.914945 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:44.915165 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 22:25:44.915327 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":119,"threads_started":1}
Aug 11 22:25:44 dist-test-slave-mvvj krb5kdc[28844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:44.927644 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:44.918582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.24.71.254:38845)
0811 22:25:44.918746 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:44.918750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:44.918813 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:25:44.918960 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:44.918963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:44.919011 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:44.919089 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:44.919809 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.920466 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.921354 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:44.921624 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:44.921712 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:44.924288 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:44.924292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:44.924294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:44.924321 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:44.925845 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.926309 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.926313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.926314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.926408 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:44.926694 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:44.926698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:44.926700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:44.926941 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:44.927099 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:44.927340 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 22:25:44.927478 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 22:25:44.928123 28894 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55896
I20260811 22:25:44.928783 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 28963
I20260811 22:25:44.938398 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 29112
I20260811 22:25:44.948345 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 29263
I20260811 22:25:44.960094 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 28859
2026-08-11T22:25:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2365 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:45 dist-test-slave-mvvj krb5kdc[29422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:45 dist-test-slave-mvvj krb5kdc[29422](info): set up 2 sockets
Aug 11 22:25:45 dist-test-slave-mvvj krb5kdc[29422](info): commencing operation
krb5kdc: starting...
W20260811 22:25:46.114437 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:46 dist-test-slave-mvvj krb5kdc[29422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:46.256032 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:39731
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:39731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:46.382057 29436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:46.382392 29436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:46.382440 29436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:46.386897 29436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:46.387006 29436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:46.387028 29436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:46.387046 29436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:46.387063 29436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:46.393428 29436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:39731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:39731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:46.394658 29436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:46.395910 29436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:46.405233 29441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:46.405233 29445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:46.405411 29436 server_base.cc:1034] running on GCE node
W20260811 22:25:46.405260 29442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:46.406234 29436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:46.407511 29436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:46.408689 29436 hybrid_clock.cc:648] HybridClock initialized: now 1786487146408669 us; error 32 us; skew 500 ppm
Aug 11 22:25:46 dist-test-slave-mvvj krb5kdc[29422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487146, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:46.411926 29436 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:46.412724 29436 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:46.412832 29436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:46.413080 29436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:46.415268 29436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "13069265784b4b1d94471d1b746b6dda"
format_stamp: "Formatted at 2026-08-11 22:25:46 on dist-test-slave-mvvj"
server_key: "9d0fbd7043dc96e75f605e9970bda937"
server_key_iv: "d8cdbb9e1ea69117b09428914a6d5a0c"
server_key_version: "encryptionkey@0"
I20260811 22:25:46.415941 29436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "13069265784b4b1d94471d1b746b6dda"
format_stamp: "Formatted at 2026-08-11 22:25:46 on dist-test-slave-mvvj"
server_key: "9d0fbd7043dc96e75f605e9970bda937"
server_key_iv: "d8cdbb9e1ea69117b09428914a6d5a0c"
server_key_version: "encryptionkey@0"
I20260811 22:25:46.420249 29436 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:25:46.423172 29453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:46.424824 29436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:46.424973 29436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "13069265784b4b1d94471d1b746b6dda"
format_stamp: "Formatted at 2026-08-11 22:25:46 on dist-test-slave-mvvj"
server_key: "9d0fbd7043dc96e75f605e9970bda937"
server_key_iv: "d8cdbb9e1ea69117b09428914a6d5a0c"
server_key_version: "encryptionkey@0"
I20260811 22:25:46.425103 29436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:46.457473 29436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:46.461158 29436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:46.461391 29436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:46.469147 29436 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:39731
I20260811 22:25:46.469446 29512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:39731 every 8 connection(s)
I20260811 22:25:46.469935 29436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:46.472967 29513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:46.474270 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 29436
I20260811 22:25:46.474417 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:46.474696 24863 external_mini_cluster.cc:1428] Setting key b725975a69f6bccd754a74b35a97831d
I20260811 22:25:46.479576 29513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda: Bootstrap starting.
I20260811 22:25:46.482136 29513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:46.483049 29513 log.cc:824] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:25:46 dist-test-slave-mvvj krb5kdc[29422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:46.485375 29513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda: No bootstrap required, opened a new log
I20260811 22:25:46.489216 29513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13069265784b4b1d94471d1b746b6dda" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39731 } }
I20260811 22:25:46.489501 29513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:46.489565 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:46.476245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33440 (local address 127.24.71.254:39731)
0811 22:25:46.476766 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:46.476782 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:46.476821 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:25:46.477492 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:46.477511 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:46.477878 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:46.478233 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:46.479140 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:46.480299 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:46.481195 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:46.481698 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:46.481878 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:46.484660 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:46.484666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:46.484683 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:46.484718 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:46.486464 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:46.486967 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:46.486972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:46.486977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:46.487119 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:46.487420 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:46.487424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:46.487426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:46.487894 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:46.488176 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:46.488426 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:25:46.488743 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":164,"threads_started":1}
I20260811 22:25:46.489579 29513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13069265784b4b1d94471d1b746b6dda, State: Initialized, Role: FOLLOWER
I20260811 22:25:46.490708 29513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13069265784b4b1d94471d1b746b6dda" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39731 } }
I20260811 22:25:46.490878 29513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:46.490962 29513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:46.491065 29513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:46.500336 29513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13069265784b4b1d94471d1b746b6dda" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39731 } }
I20260811 22:25:46.500819 29513 leader_election.cc:302] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13069265784b4b1d94471d1b746b6dda; no voters:
I20260811 22:25:46.501232 29513 leader_election.cc:288] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:46.501398 29518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:46.501690 29518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [term 1 LEADER]: Becoming Leader. State: Replica: 13069265784b4b1d94471d1b746b6dda, State: Running, Role: LEADER
I20260811 22:25:46.502123 29518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13069265784b4b1d94471d1b746b6dda" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39731 } }
I20260811 22:25:46.502825 29513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:46.504176 29519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13069265784b4b1d94471d1b746b6dda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13069265784b4b1d94471d1b746b6dda" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39731 } } }
I20260811 22:25:46.504333 29519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:46.504768 29520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13069265784b4b1d94471d1b746b6dda. Latest consensus state: current_term: 1 leader_uuid: "13069265784b4b1d94471d1b746b6dda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13069265784b4b1d94471d1b746b6dda" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 39731 } } }
I20260811 22:25:46.504976 29520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda [sys.catalog]: This master's current role is: LEADER
W20260811 22:25:46.508571 29534 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:25:46.508661 29534 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:25:46.508745 29528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:46.509897 29528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:46.515987 29528 catalog_manager.cc:1347] Generated new cluster ID: 8aadc6a02fde48a49eadd48f15460015
I20260811 22:25:46.516114 29528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:46.546576 29528 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:46.548170 29528 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:46.558636 29528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda: Generated new TSK 0
I20260811 22:25:46.559499 29528 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
W20260811 22:25:47.378544 24863 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.71.193: real 0.806s user 0.002s sys 0.003s
I20260811 22:25:47.379109 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39731
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:47.499744 29540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:47.500083 29540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:47.500165 29540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:47.504550 29540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:47.504628 29540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:47.504693 29540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:47.510264 29540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:47.511453 29540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:47.512580 29540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:47.522779 29549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:47.522787 29547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:47.522787 29546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:47.523118 29540 server_base.cc:1034] running on GCE node
I20260811 22:25:47.523658 29540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:47.524472 29540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:47.525672 29540 hybrid_clock.cc:648] HybridClock initialized: now 1786487147525638 us; error 47 us; skew 500 ppm
Aug 11 22:25:47 dist-test-slave-mvvj krb5kdc[29422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:47.538307 29540 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:25:47.539232 29540 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:47.539340 29540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:47.539598 29540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:47.541970 29540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "d2cbb039c0eb41f181321d33ebb7718c"
format_stamp: "Formatted at 2026-08-11 22:25:47 on dist-test-slave-mvvj"
server_key: "995234ef2a77ba1d06e34b800518fe70"
server_key_iv: "fe36d44dbd92a1295cf5dad933c6a4b0"
server_key_version: "encryptionkey@0"
I20260811 22:25:47.542630 29540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "d2cbb039c0eb41f181321d33ebb7718c"
format_stamp: "Formatted at 2026-08-11 22:25:47 on dist-test-slave-mvvj"
server_key: "995234ef2a77ba1d06e34b800518fe70"
server_key_iv: "fe36d44dbd92a1295cf5dad933c6a4b0"
server_key_version: "encryptionkey@0"
I20260811 22:25:47.548452 29540 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 22:25:47.551395 29557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:47.553040 29540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:47.553200 29540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "d2cbb039c0eb41f181321d33ebb7718c"
format_stamp: "Formatted at 2026-08-11 22:25:47 on dist-test-slave-mvvj"
server_key: "995234ef2a77ba1d06e34b800518fe70"
server_key_iv: "fe36d44dbd92a1295cf5dad933c6a4b0"
server_key_version: "encryptionkey@0"
I20260811 22:25:47.553360 29540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:47.576713 29540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:47.580585 29540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:47.580806 29540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:47.581485 29540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:47.582479 29540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:47.582553 29540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:47.582626 29540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:47.582664 29540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:47.593384 29540 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:42597
I20260811 22:25:47.593405 29684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:42597 every 8 connection(s)
I20260811 22:25:47.594149 29540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:47.597568 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 29540
I20260811 22:25:47.597736 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:47.598172 24863 external_mini_cluster.cc:1428] Setting key b3781ec5005d90372cc961aa2f32d45a
Aug 11 22:25:47 dist-test-slave-mvvj krb5kdc[29422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487147, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:25:47.613844 29687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:47.596371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:38505 (local address 127.24.71.254:39731)
0811 22:25:47.596748 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:47.596752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:47.597798 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 22:25:47.599549 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:47.599553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:47.599601 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:47.599707 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:47.601710 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:47.602360 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:47.603999 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:47.604512 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:47.604653 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:47.608111 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:47.608118 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:47.608132 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:47.608179 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:47.611318 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:47.612107 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:47.612111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:47.612112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:47.612271 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:47.612679 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:47.612682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:47.612683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:47.612891 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:47.613010 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:47.613562 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 22:25:47.613685 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":181,"threads_started":1}
I20260811 22:25:47.614773 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:47.596773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:39731 (local address 127.24.71.193:38505)
0811 22:25:47.597357 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:47.597433 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:47.599301 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:47.599979 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:47.599990 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:47.600657 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:47.601511 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:47.601538 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:47.602588 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:47.602592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:47.603784 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:47.603796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:47.604689 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:47.604692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:47.604878 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:47.605642 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:47.605675 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:47.607786 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:47.611522 (+ 3736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:47.611535 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:47.611554 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:47.611986 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:47.612396 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:47.612399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:47.612401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:47.612575 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:47.613023 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:47.613029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:47.613303 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 22:25:47.613487 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 22:25:47.613786 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":135,"threads_started":1}
I20260811 22:25:47.616312 29685 heartbeater.cc:347] Connected to a master server at 127.24.71.254:39731
I20260811 22:25:47.616581 29685 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:47.617081 29685 heartbeater.cc:511] Master 127.24.71.254:39731 requested a full tablet report, sending...
I20260811 22:25:47.618671 29471 ts_manager.cc:194] Registered new tserver with Master: d2cbb039c0eb41f181321d33ebb7718c (127.24.71.193:42597)
I20260811 22:25:47.619813 29471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:38505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
W20260811 22:25:48.192690 24863 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.71.194: real 0.592s user 0.002s sys 0.003s
I20260811 22:25:48.193349 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39731
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:48.311398 29691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:48.311729 29691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:48.311818 29691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:48.316035 29691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:48.316150 29691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:48.316250 29691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:48.322086 29691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:48.323272 29691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:48.324430 29691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:48.334398 29696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:48.334352 29697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:48.334520 29691 server_base.cc:1034] running on GCE node
W20260811 22:25:48.334326 29699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:48.335063 29691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:48.335785 29691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:48.337038 29691 hybrid_clock.cc:648] HybridClock initialized: now 1786487148337022 us; error 32 us; skew 500 ppm
Aug 11 22:25:48 dist-test-slave-mvvj krb5kdc[29422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487148, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:48.340129 29691 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:25:48.340934 29691 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:48.341001 29691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:48.341176 29691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:48.343281 29691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "c90984f1e43c4aa0984b9a81aceb2fae"
format_stamp: "Formatted at 2026-08-11 22:25:48 on dist-test-slave-mvvj"
server_key: "e3ca35f8fb9bb9986e2cf7de1ba441b5"
server_key_iv: "7108cd9e1843cb601e0472a136194975"
server_key_version: "encryptionkey@0"
I20260811 22:25:48.343814 29691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "c90984f1e43c4aa0984b9a81aceb2fae"
format_stamp: "Formatted at 2026-08-11 22:25:48 on dist-test-slave-mvvj"
server_key: "e3ca35f8fb9bb9986e2cf7de1ba441b5"
server_key_iv: "7108cd9e1843cb601e0472a136194975"
server_key_version: "encryptionkey@0"
I20260811 22:25:48.347713 29691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:25:48.350281 29705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:48.351958 29691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:48.352109 29691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "c90984f1e43c4aa0984b9a81aceb2fae"
format_stamp: "Formatted at 2026-08-11 22:25:48 on dist-test-slave-mvvj"
server_key: "e3ca35f8fb9bb9986e2cf7de1ba441b5"
server_key_iv: "7108cd9e1843cb601e0472a136194975"
server_key_version: "encryptionkey@0"
I20260811 22:25:48.352240 29691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:48.372692 29691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:48.376618 29691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:48.376868 29691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:48.377554 29691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:48.378527 29691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:48.378607 29691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:48.378687 29691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:48.378732 29691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:48.389050 29691 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:46803
I20260811 22:25:48.389070 29830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:46803 every 8 connection(s)
I20260811 22:25:48.389735 29691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:48.391426 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 29691
I20260811 22:25:48.391561 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:48.391948 24863 external_mini_cluster.cc:1428] Setting key c9e01fd2d1b193b24406ddf4318e6b9f
Aug 11 22:25:48 dist-test-slave-mvvj krb5kdc[29422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487148, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:25:48.407408 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.391633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:53781 (local address 127.24.71.254:39731)
0811 22:25:48.391951 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.391957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.392992 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 22:25:48.395037 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.395040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.395090 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.395183 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:48.396879 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.397486 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.398944 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.399208 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.399336 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.402450 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:48.402453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:48.402456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:48.402484 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:48.404645 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.405422 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.405426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.405427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.405564 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.406041 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.406044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.406045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.406259 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:48.406344 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.407094 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.407209 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":79,"threads_started":1}
I20260811 22:25:48.408514 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.392155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:39731 (local address 127.24.71.194:53781)
0811 22:25:48.392822 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:48.392883 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:48.394745 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:48.395401 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:48.395414 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:48.396094 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:48.396701 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.396717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.397641 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.397651 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.398789 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.398801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.399396 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.399400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.399591 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.400366 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:48.400426 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:48.402169 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:48.404791 (+ 2622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:48.404799 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:48.404821 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:48.405222 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:48.405726 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:48.405731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:48.405733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:48.405917 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:48.406372 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:48.406431 (+ 59us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:48.406814 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 22:25:48.407036 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 22:25:48.407449 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":178,"threads_started":1}
I20260811 22:25:48.409763 29831 heartbeater.cc:347] Connected to a master server at 127.24.71.254:39731
I20260811 22:25:48.410040 29831 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:48.410601 29831 heartbeater.cc:511] Master 127.24.71.254:39731 requested a full tablet report, sending...
I20260811 22:25:48.411716 29471 ts_manager.cc:194] Registered new tserver with Master: c90984f1e43c4aa0984b9a81aceb2fae (127.24.71.194:46803)
I20260811 22:25:48.412334 29471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:53781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:25:48.424364 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39731
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:48.544418 29838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:48.544698 29838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:48.544744 29838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:48.548700 29838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:48.548763 29838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:48.548821 29838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:48.555558 29838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:39731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:48.556833 29838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:48.557966 29838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:48.568279 29847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:48.568352 29838 server_base.cc:1034] running on GCE node
W20260811 22:25:48.568336 29845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:48.568284 29843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:48.568950 29838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:48.569691 29838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:48.570909 29838 hybrid_clock.cc:648] HybridClock initialized: now 1786487148570873 us; error 47 us; skew 500 ppm
Aug 11 22:25:48 dist-test-slave-mvvj krb5kdc[29422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487148, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:48.574149 29838 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:25:48.575150 29838 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:48.575260 29838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:48.575532 29838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:48.577885 29838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "4ac6575250c5479aad5c41a301675502"
format_stamp: "Formatted at 2026-08-11 22:25:48 on dist-test-slave-mvvj"
server_key: "3c645cff413200be366cb2e710cdf8f2"
server_key_iv: "120ecf1dbea2280d243bf274df93bf61"
server_key_version: "encryptionkey@0"
I20260811 22:25:48.578552 29838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "4ac6575250c5479aad5c41a301675502"
format_stamp: "Formatted at 2026-08-11 22:25:48 on dist-test-slave-mvvj"
server_key: "3c645cff413200be366cb2e710cdf8f2"
server_key_iv: "120ecf1dbea2280d243bf274df93bf61"
server_key_version: "encryptionkey@0"
I20260811 22:25:48.582885 29838 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:48.585582 29855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:48.587059 29838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:48.587245 29838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "4ac6575250c5479aad5c41a301675502"
format_stamp: "Formatted at 2026-08-11 22:25:48 on dist-test-slave-mvvj"
server_key: "3c645cff413200be366cb2e710cdf8f2"
server_key_iv: "120ecf1dbea2280d243bf274df93bf61"
server_key_version: "encryptionkey@0"
I20260811 22:25:48.587401 29838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:48.600965 29838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:48.604715 29838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:48.604938 29838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:48.605613 29838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:48.606509 29838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:48.606591 29838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:48.606671 29838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:48.606724 29838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:48.616427 29838 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:33259
I20260811 22:25:48.616456 29978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:33259 every 8 connection(s)
I20260811 22:25:48.617122 29838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:48.622165 29685 heartbeater.cc:503] Master 127.24.71.254:39731 was elected leader, sending a full tablet report...
I20260811 22:25:48.623216 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 29838
I20260811 22:25:48.623360 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:48.623725 24863 external_mini_cluster.cc:1428] Setting key 164e76d56b182a941c4698cd3ae7d2d8
Aug 11 22:25:48 dist-test-slave-mvvj krb5kdc[29422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487148, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:48.631392 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.619014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:40283 (local address 127.24.71.254:39731)
0811 22:25:48.619165 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.619169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.620089 (+ 920us) server_negotiation.cc:407] Connection header received
0811 22:25:48.621457 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.621460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.621518 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.621601 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:48.622831 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.623504 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.624758 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.625097 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.625206 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.627810 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:48.627813 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:48.627816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:48.627874 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:48.629333 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.629820 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.629823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.629824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.629901 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.630246 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.630249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.630250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.630501 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:48.630593 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.631117 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.631233 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:25:48.632263 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.619422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:39731 (local address 127.24.71.195:40283)
0811 22:25:48.619939 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:48.619984 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:48.621226 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:48.621771 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:48.621778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:48.622132 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:48.622677 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.622693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.623677 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.623680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.624652 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.624663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.625269 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.625273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.625470 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.626179 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:48.626196 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:48.627651 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:48.629443 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:48.629448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:48.629460 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:48.629729 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:48.629990 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:48.629992 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:48.629993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:48.630124 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:48.630601 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:48.630606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:48.630831 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 22:25:48.631044 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 22:25:48.631302 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":116,"threads_started":1}
I20260811 22:25:48.633572 29979 heartbeater.cc:347] Connected to a master server at 127.24.71.254:39731
I20260811 22:25:48.633828 29979 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:48.634300 29979 heartbeater.cc:511] Master 127.24.71.254:39731 requested a full tablet report, sending...
I20260811 22:25:48.635425 29471 ts_manager.cc:194] Registered new tserver with Master: 4ac6575250c5479aad5c41a301675502 (127.24.71.195:33259)
I20260811 22:25:48.636065 29471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:40283
I20260811 22:25:48.637944 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:48.647565 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.640016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.24.71.254:39731)
0811 22:25:48.640150 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.640157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.640391 (+ 234us) server_negotiation.cc:407] Connection header received
0811 22:25:48.640507 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.640510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.640564 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.640644 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:48.641455 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.641991 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.642869 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.643091 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.643172 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.644653 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:48.644655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:48.644657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:48.644680 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:48.645944 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.646348 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.646350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.646351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.646479 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:48.646761 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:48.646763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:48.646764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:48.646974 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:48.647081 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.647298 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.647444 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:25:48.650352 29471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:48.653486 29471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:48.669234 29996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.663704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.24.71.193:42597)
0811 22:25:48.664297 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.664304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.664330 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:25:48.664411 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.664416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.664617 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.665123 (+ 506us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:48.665250 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.665958 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.667575 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.668506 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.668624 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.668777 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.669027 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.669117 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":168,"threads_started":1}
I20260811 22:25:48.669250 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.662695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:42597 (local address 127.0.0.1:45414)
0811 22:25:48.663533 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:48.663605 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:48.663716 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:48.664773 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:48.664776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:48.664810 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:48.664965 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.664972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.666075 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.666081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.667439 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.667451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.668640 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.668645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.668779 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.668830 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:25:48.668956 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 22:25:48.669087 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":536,"spinlock_wait_cycles":4864,"thread_start_us":88,"threads_started":1}
I20260811 22:25:48.670080 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.662694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33259 (local address 127.0.0.1:52588)
0811 22:25:48.663163 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:48.663214 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:48.663357 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:48.664759 (+ 1402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:48.664765 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:48.664787 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:48.664956 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.664964 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.666452 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.666455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.667550 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.667561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.669644 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.669648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.669752 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.669780 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:25:48.669847 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:25:48.669929 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"spinlock_wait_cycles":2304,"thread_start_us":140,"threads_started":1}
I20260811 22:25:48.670292 29993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.662887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52588 (local address 127.24.71.195:33259)
0811 22:25:48.664327 (+ 1440us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.664340 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.664364 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:25:48.664450 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.664454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.664603 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.664765 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:48.665072 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.666298 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.667665 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.668506 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.668624 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.668777 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.670031 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.670139 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1332,"thread_start_us":169,"threads_started":1}
I20260811 22:25:48.670444 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.663795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60490 (local address 127.24.71.194:46803)
0811 22:25:48.664322 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.664332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.664351 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:25:48.664425 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.664429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.664617 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.664765 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:48.665171 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.666300 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.668835 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.669683 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.669785 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.669936 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.670173 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.670275 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":192,"threads_started":1}
I20260811 22:25:48.670696 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.663635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46803 (local address 127.0.0.1:60490)
0811 22:25:48.663993 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:48.664009 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:48.664094 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:48.664933 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:48.664937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:48.664952 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:48.665044 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.665053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.667597 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.667601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.668689 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.668698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.669810 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.669814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.669914 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.669940 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:25:48.670020 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:25:48.670087 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":105,"threads_started":1}
I20260811 22:25:48.672775 29611 tablet_service.cc:1467] Processing CreateTablet for tablet b47e8f478b07410a937fb6bd1c032220 (DEFAULT_TABLE table=test-table [id=4fb218c640174e23b34a2656586b0a3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:48.673820 29611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b47e8f478b07410a937fb6bd1c032220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:48.673549 29903 tablet_service.cc:1467] Processing CreateTablet for tablet b47e8f478b07410a937fb6bd1c032220 (DEFAULT_TABLE table=test-table [id=4fb218c640174e23b34a2656586b0a3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:48.673547 29757 tablet_service.cc:1467] Processing CreateTablet for tablet b47e8f478b07410a937fb6bd1c032220 (DEFAULT_TABLE table=test-table [id=4fb218c640174e23b34a2656586b0a3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:48.674582 29757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b47e8f478b07410a937fb6bd1c032220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:48.674595 29903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b47e8f478b07410a937fb6bd1c032220. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:48.680356 29998 tablet_bootstrap.cc:492] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c: Bootstrap starting.
I20260811 22:25:48.682484 29999 tablet_bootstrap.cc:492] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: Bootstrap starting.
I20260811 22:25:48.683290 29998 tablet_bootstrap.cc:654] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:48.684301 29998 log.cc:824] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:48.681979 30000 tablet_bootstrap.cc:492] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: Bootstrap starting.
I20260811 22:25:48.684942 29999 tablet_bootstrap.cc:654] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:48.686236 29999 log.cc:824] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:48.686295 29998 tablet_bootstrap.cc:492] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c: No bootstrap required, opened a new log
I20260811 22:25:48.686561 29998 ts_tablet_manager.cc:1397] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:48.687098 30000 tablet_bootstrap.cc:654] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:48.688213 30000 log.cc:824] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:48.688463 29999 tablet_bootstrap.cc:492] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: No bootstrap required, opened a new log
I20260811 22:25:48.688657 29999 ts_tablet_manager.cc:1397] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:25:48.690287 30000 tablet_bootstrap.cc:492] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: No bootstrap required, opened a new log
I20260811 22:25:48.690477 30000 ts_tablet_manager.cc:1397] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260811 22:25:48.691181 29998 raft_consensus.cc:348] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } }
I20260811 22:25:48.691498 29998 raft_consensus.cc:374] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:48.691586 29998 raft_consensus.cc:729] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2cbb039c0eb41f181321d33ebb7718c, State: Initialized, Role: FOLLOWER
I20260811 22:25:48.692097 29998 consensus_queue.cc:260] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } }
I20260811 22:25:48.692719 29999 raft_consensus.cc:348] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } }
I20260811 22:25:48.693053 29999 raft_consensus.cc:374] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:48.693147 29999 raft_consensus.cc:729] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90984f1e43c4aa0984b9a81aceb2fae, State: Initialized, Role: FOLLOWER
I20260811 22:25:48.693343 29998 ts_tablet_manager.cc:1428] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:25:48.693655 29999 consensus_queue.cc:260] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } }
I20260811 22:25:48.694000 30000 raft_consensus.cc:348] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } }
I20260811 22:25:48.694283 30000 raft_consensus.cc:374] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:48.694341 30000 raft_consensus.cc:729] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac6575250c5479aad5c41a301675502, State: Initialized, Role: FOLLOWER
I20260811 22:25:48.694687 29999 ts_tablet_manager.cc:1428] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:25:48.694756 30000 consensus_queue.cc:260] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } }
I20260811 22:25:48.695060 29831 heartbeater.cc:503] Master 127.24.71.254:39731 was elected leader, sending a full tablet report...
I20260811 22:25:48.695454 29979 heartbeater.cc:503] Master 127.24.71.254:39731 was elected leader, sending a full tablet report...
I20260811 22:25:48.695853 30000 ts_tablet_manager.cc:1428] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:25:48.810235 30007 raft_consensus.cc:482] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:48.810503 30007 raft_consensus.cc:504] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } }
I20260811 22:25:48.811694 30007 leader_election.cc:288] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ac6575250c5479aad5c41a301675502 (127.24.71.195:33259), d2cbb039c0eb41f181321d33ebb7718c (127.24.71.193:42597)
I20260811 22:25:48.815289 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.811966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33259 (local address 127.24.71.194:39265)
0811 22:25:48.812092 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:48.812109 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:48.812236 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:48.812532 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:48.812535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:48.812552 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:48.812673 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.812678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.813393 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.813399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.814187 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.814195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.814845 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.814848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.814969 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.814988 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:25:48.815079 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:25:48.815149 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":2}
I20260811 22:25:48.815289 29993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.812056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:39265 (local address 127.24.71.195:33259)
0811 22:25:48.812220 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.812226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.812251 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:25:48.812331 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.812333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.812414 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.812519 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:48.812786 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.813279 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.814322 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.814721 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.814848 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.814903 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.815096 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.815166 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:25:48.815819 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.811978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:42597 (local address 127.24.71.194:59955)
0811 22:25:48.812675 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:48.812687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:48.812808 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:48.813076 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:48.813079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:48.813095 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:48.813215 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.813221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.813929 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.813933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.814703 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:48.814711 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.815398 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.815402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:48.815505 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.815527 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:25:48.815600 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:25:48.815665 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":346,"thread_start_us":142,"threads_started":1}
I20260811 22:25:48.815902 29996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.812148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:59955 (local address 127.24.71.193:42597)
0811 22:25:48.812316 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.812322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.812778 (+ 456us) server_negotiation.cc:407] Connection header received
0811 22:25:48.812926 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.812929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.812978 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.813054 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:48.813326 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.813804 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.814838 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.815249 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.815392 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.815441 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.815644 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.815728 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:25:48.816160 29929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b47e8f478b07410a937fb6bd1c032220" candidate_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac6575250c5479aad5c41a301675502" is_pre_election: true
I20260811 22:25:48.816490 29636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b47e8f478b07410a937fb6bd1c032220" candidate_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cbb039c0eb41f181321d33ebb7718c" is_pre_election: true
I20260811 22:25:48.816545 29929 raft_consensus.cc:2454] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c90984f1e43c4aa0984b9a81aceb2fae in term 0.
I20260811 22:25:48.816756 29636 raft_consensus.cc:2454] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c90984f1e43c4aa0984b9a81aceb2fae in term 0.
I20260811 22:25:48.817101 29707 leader_election.cc:302] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac6575250c5479aad5c41a301675502, c90984f1e43c4aa0984b9a81aceb2fae; no voters:
I20260811 22:25:48.817445 30007 raft_consensus.cc:2781] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:48.817521 30007 raft_consensus.cc:482] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:48.817553 30007 raft_consensus.cc:3037] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:48.818645 30007 raft_consensus.cc:504] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } }
I20260811 22:25:48.819044 30007 leader_election.cc:288] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [CANDIDATE]: Term 1 election: Requested vote from peers 4ac6575250c5479aad5c41a301675502 (127.24.71.195:33259), d2cbb039c0eb41f181321d33ebb7718c (127.24.71.193:42597)
I20260811 22:25:48.819375 29929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b47e8f478b07410a937fb6bd1c032220" candidate_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac6575250c5479aad5c41a301675502"
I20260811 22:25:48.819525 29929 raft_consensus.cc:3037] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:48.819555 29636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b47e8f478b07410a937fb6bd1c032220" candidate_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2cbb039c0eb41f181321d33ebb7718c"
I20260811 22:25:48.819739 29636 raft_consensus.cc:3037] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:48.820674 29929 raft_consensus.cc:2454] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c90984f1e43c4aa0984b9a81aceb2fae in term 1.
I20260811 22:25:48.820914 29636 raft_consensus.cc:2454] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c90984f1e43c4aa0984b9a81aceb2fae in term 1.
I20260811 22:25:48.820996 29707 leader_election.cc:302] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac6575250c5479aad5c41a301675502, c90984f1e43c4aa0984b9a81aceb2fae; no voters:
I20260811 22:25:48.821192 30007 raft_consensus.cc:2781] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:48.821496 30007 raft_consensus.cc:686] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 1 LEADER]: Becoming Leader. State: Replica: c90984f1e43c4aa0984b9a81aceb2fae, State: Running, Role: LEADER
I20260811 22:25:48.821871 30007 consensus_queue.cc:237] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } }
I20260811 22:25:48.825801 29470 catalog_manager.cc:5495] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae reported cstate change: term changed from 0 to 1, leader changed from <none> to c90984f1e43c4aa0984b9a81aceb2fae (127.24.71.194). New cstate: current_term: 1 leader_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c90984f1e43c4aa0984b9a81aceb2fae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46803 } health_report { overall_health: HEALTHY } } }
W20260811 22:25:48.846138 29686 tablet.cc:2369] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:25:48.868192 29980 tablet.cc:2369] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:48.873216 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.868937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.24.71.194:46803)
0811 22:25:48.869106 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:48.869111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:48.869128 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:25:48.869255 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:48.869259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:48.869334 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:48.869462 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:48.869773 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.870352 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.871324 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:48.871576 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:48.871701 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:48.872094 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:48.872754 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:48.872914 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:48.872998 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 22:25:48.873062 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:25:48.879160 29636 raft_consensus.cc:1261] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c [term 1 FOLLOWER]: Refusing update from remote peer c90984f1e43c4aa0984b9a81aceb2fae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:48.879160 29929 raft_consensus.cc:1261] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [term 1 FOLLOWER]: Refusing update from remote peer c90984f1e43c4aa0984b9a81aceb2fae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:48.879997 30011 consensus_queue.cc:1035] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac6575250c5479aad5c41a301675502" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:48.880175 30007 consensus_queue.cc:1035] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2cbb039c0eb41f181321d33ebb7718c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:48.888788 30016 mvcc.cc:205] Tried to move back new op lower bound from 7317451361801826304 to 7317451361577947136. Current Snapshot: MvccSnapshot[applied={T|T < 7317451361801826304}]
I20260811 22:25:48.896659 29471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:48.896821 29471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:48.899376 29471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13069265784b4b1d94471d1b746b6dda: Sending DeleteTablet for 3 replicas of tablet b47e8f478b07410a937fb6bd1c032220
I20260811 22:25:48.900290 29903 tablet_service.cc:1514] Processing DeleteTablet for tablet b47e8f478b07410a937fb6bd1c032220 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:48 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:52588
I20260811 22:25:48.900291 29611 tablet_service.cc:1514] Processing DeleteTablet for tablet b47e8f478b07410a937fb6bd1c032220 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:48 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:45414
I20260811 22:25:48.900426 29757 tablet_service.cc:1514] Processing DeleteTablet for tablet b47e8f478b07410a937fb6bd1c032220 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:48 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60490
I20260811 22:25:48.900787 30025 tablet_replica.cc:330] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: stopping tablet replica
I20260811 22:25:48.901029 30026 tablet_replica.cc:330] T b47e8f478b07410a937fb6bd1c032220 P d2cbb039c0eb41f181321d33ebb7718c: stopping tablet replica
I20260811 22:25:48.901188 30025 raft_consensus.cc:2229] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:48.901260 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 29540
I20260811 22:25:48.901340 30027 tablet_replica.cc:330] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: stopping tablet replica
I20260811 22:25:48.901547 30025 raft_consensus.cc:2258] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:48.901968 30027 raft_consensus.cc:2229] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:48.902233 30027 raft_consensus.cc:2258] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:48.904443 30025 ts_tablet_manager.cc:1905] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:48.905478 30027 ts_tablet_manager.cc:1905] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:48.908066 30025 ts_tablet_manager.cc:1918] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:48.908214 30025 log.cc:1196] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/b47e8f478b07410a937fb6bd1c032220
I20260811 22:25:48.908229 30027 ts_tablet_manager.cc:1918] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:48.908385 30027 log.cc:1196] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/b47e8f478b07410a937fb6bd1c032220
I20260811 22:25:48.908607 30025 ts_tablet_manager.cc:1939] T b47e8f478b07410a937fb6bd1c032220 P 4ac6575250c5479aad5c41a301675502: Deleting consensus metadata
I20260811 22:25:48.908731 30027 ts_tablet_manager.cc:1939] T b47e8f478b07410a937fb6bd1c032220 P c90984f1e43c4aa0984b9a81aceb2fae: Deleting consensus metadata
I20260811 22:25:48.909976 29454 catalog_manager.cc:4841] TS 4ac6575250c5479aad5c41a301675502 (127.24.71.195:33259): tablet b47e8f478b07410a937fb6bd1c032220 (table test-table [id=4fb218c640174e23b34a2656586b0a3f]) successfully deleted
I20260811 22:25:48.910001 29456 catalog_manager.cc:4841] TS c90984f1e43c4aa0984b9a81aceb2fae (127.24.71.194:46803): tablet b47e8f478b07410a937fb6bd1c032220 (table test-table [id=4fb218c640174e23b34a2656586b0a3f]) successfully deleted
W20260811 22:25:48.912230 29456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:42597 (error 108)
I20260811 22:25:48.913157 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 29691
I20260811 22:25:48.913383 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:48.913031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:42597 (local address 127.0.0.1:45418)
0811 22:25:48.913187 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:48.913276 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:42597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 22:25:48.916234 29456 catalog_manager.cc:4584] TS d2cbb039c0eb41f181321d33ebb7718c (127.24.71.193:42597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b47e8f478b07410a937fb6bd1c032220: Network error: Client connection negotiation failed: client connection to 127.24.71.193:42597: connect: Connection refused (error 111)
I20260811 22:25:48.922895 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 29838
I20260811 22:25:48.930756 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 29436
2026-08-11T22:25:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3969 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T22:25:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:48Z Disabled control of system clock
I20260811 22:25:48.966634 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:45885
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:45885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:25:49.088809 30032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:49.089075 30032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:49.089112 30032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:49.093194 30032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:49.093261 30032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:49.093277 30032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:49.093294 30032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:49.093309 30032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:49.098187 30032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:45885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:45885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:49.099359 30032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:49.100811 30032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:49.109483 30038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:49.109531 30032 server_base.cc:1034] running on GCE node
W20260811 22:25:49.109475 30039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:49.109479 30041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:49.110251 30032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:49.111405 30032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:49.112627 30032 hybrid_clock.cc:648] HybridClock initialized: now 1786487149112610 us; error 29 us; skew 500 ppm
I20260811 22:25:49.114933 30032 webserver.cc:460] Webserver started at http://127.24.71.254:45547/ using document root <none> and password file <none>
I20260811 22:25:49.115540 30032 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:49.115641 30032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:49.115923 30032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:49.118079 30032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "2363d81d1a2b4a5099ad0ea396f14d21"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "701dfa6e350bda1601cdd21ffb4b0fc2"
server_key_iv: "3a875c5601ce08127e1388eca19b271d"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.118693 30032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "2363d81d1a2b4a5099ad0ea396f14d21"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "701dfa6e350bda1601cdd21ffb4b0fc2"
server_key_iv: "3a875c5601ce08127e1388eca19b271d"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.122691 30032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:49.125277 30047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.126626 30032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:25:49.126780 30032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "2363d81d1a2b4a5099ad0ea396f14d21"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "701dfa6e350bda1601cdd21ffb4b0fc2"
server_key_iv: "3a875c5601ce08127e1388eca19b271d"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.126910 30032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:49.131673 30032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:49.132375 30032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:49.132563 30032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:49.139809 30032 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:45885
I20260811 22:25:49.139819 30106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:45885 every 8 connection(s)
I20260811 22:25:49.140844 30032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:49.144276 30107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:49.149582 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 30032
I20260811 22:25:49.149725 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:49.150081 24863 external_mini_cluster.cc:1428] Setting key 5a37d0441f21f03c2be7f835d16125e8
I20260811 22:25:49.150995 30107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21: Bootstrap starting.
I20260811 22:25:49.153957 30107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:49.154924 30107 log.cc:824] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:49.156702 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.151674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54190 (local address 127.24.71.254:45885)
0811 22:25:49.152468 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.152491 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.152537 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:25:49.153397 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.153420 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.153888 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.154283 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.155093 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.155099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.155114 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.155126 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.155195 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.155220 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.155345 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.155486 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.155756 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":190,"threads_started":1}
I20260811 22:25:49.157418 30107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21: No bootstrap required, opened a new log
I20260811 22:25:49.161216 30107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2363d81d1a2b4a5099ad0ea396f14d21" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45885 } }
I20260811 22:25:49.161442 30107 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:49.161477 30107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2363d81d1a2b4a5099ad0ea396f14d21, State: Initialized, Role: FOLLOWER
I20260811 22:25:49.161890 30107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2363d81d1a2b4a5099ad0ea396f14d21" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45885 } }
I20260811 22:25:49.162010 30107 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:49.162056 30107 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:49.162128 30107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:49.162982 30107 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2363d81d1a2b4a5099ad0ea396f14d21" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45885 } }
I20260811 22:25:49.163286 30107 leader_election.cc:302] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2363d81d1a2b4a5099ad0ea396f14d21; no voters:
I20260811 22:25:49.163550 30107 leader_election.cc:288] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:49.163689 30113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:49.164093 30113 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [term 1 LEADER]: Becoming Leader. State: Replica: 2363d81d1a2b4a5099ad0ea396f14d21, State: Running, Role: LEADER
I20260811 22:25:49.164505 30113 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2363d81d1a2b4a5099ad0ea396f14d21" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45885 } }
I20260811 22:25:49.165002 30107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:49.166384 30114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2363d81d1a2b4a5099ad0ea396f14d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2363d81d1a2b4a5099ad0ea396f14d21" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45885 } } }
I20260811 22:25:49.166368 30115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2363d81d1a2b4a5099ad0ea396f14d21. Latest consensus state: current_term: 1 leader_uuid: "2363d81d1a2b4a5099ad0ea396f14d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2363d81d1a2b4a5099ad0ea396f14d21" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45885 } } }
I20260811 22:25:49.166541 30114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:49.166541 30115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:49.166985 30124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:49.170084 30124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:49.175316 30124 catalog_manager.cc:1347] Generated new cluster ID: 3a1ebf2314b1442faca62b0139729147
I20260811 22:25:49.175400 30124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:49.193118 30124 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:49.193490 30124 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:49.201637 30124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21: Generated new TSK 0
I20260811 22:25:49.202606 30124 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:49.212533 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45885
--builtin_ntp_servers=127.24.71.212:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:25:49.338268 30135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:49.338598 30135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:49.338672 30135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:49.342754 30135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:49.342836 30135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:49.342931 30135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:49.347726 30135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:49.348889 30135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:49.350315 30135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:49.360870 30140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:49.361022 30141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:49.360870 30143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:49.361523 30135 server_base.cc:1034] running on GCE node
I20260811 22:25:49.361975 30135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:49.362659 30135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:49.363925 30135 hybrid_clock.cc:648] HybridClock initialized: now 1786487149363888 us; error 27 us; skew 500 ppm
I20260811 22:25:49.366209 30135 webserver.cc:460] Webserver started at http://127.24.71.193:46649/ using document root <none> and password file <none>
I20260811 22:25:49.366937 30135 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:49.367044 30135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:49.367291 30135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:49.369860 30135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "95f163f1ea754892b086a22474ba810c"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "f8b519154539b45edcd1fed75c274fa6"
server_key_iv: "0cbee40e3217cd476eb810b204cf5438"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.370594 30135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "95f163f1ea754892b086a22474ba810c"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "f8b519154539b45edcd1fed75c274fa6"
server_key_iv: "0cbee40e3217cd476eb810b204cf5438"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.374922 30135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 22:25:49.377854 30149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.379194 30135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:49.379348 30135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "95f163f1ea754892b086a22474ba810c"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "f8b519154539b45edcd1fed75c274fa6"
server_key_iv: "0cbee40e3217cd476eb810b204cf5438"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.379685 30135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:49.384737 30135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:49.385406 30135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:49.385603 30135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:49.386180 30135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:49.387192 30135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:49.387267 30135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.387348 30135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:49.387396 30135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.396651 30135 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:38979
I20260811 22:25:49.396673 30276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:38979 every 8 connection(s)
I20260811 22:25:49.397648 30135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:49.400745 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 30135
I20260811 22:25:49.400867 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:49.401165 24863 external_mini_cluster.cc:1428] Setting key d29f333f6f139e74f6fbd4fd760d658c
I20260811 22:25:49.403508 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45885
--builtin_ntp_servers=127.24.71.212:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:25:49.404507 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.399571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:47267 (local address 127.24.71.254:45885)
0811 22:25:49.399717 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.399721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.400847 (+ 1126us) server_negotiation.cc:407] Connection header received
0811 22:25:49.402521 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.402524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.402568 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.402651 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.403999 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.404002 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.404005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.404027 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.404049 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.404058 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.404159 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.404313 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.404362 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:25:49.405429 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.400009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:45885 (local address 127.24.71.193:47267)
0811 22:25:49.400683 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:49.400725 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:49.402295 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:49.402827 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:49.402834 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:49.402939 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:49.403628 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:49.403632 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:49.403702 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:49.403794 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:49.404143 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:49.404153 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:49.404159 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:25:49.404361 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 22:25:49.404624 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":111,"threads_started":1}
I20260811 22:25:49.406788 30277 heartbeater.cc:347] Connected to a master server at 127.24.71.254:45885
I20260811 22:25:49.407055 30277 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:49.407500 30277 heartbeater.cc:511] Master 127.24.71.254:45885 requested a full tablet report, sending...
I20260811 22:25:49.409387 30066 ts_manager.cc:194] Registered new tserver with Master: 95f163f1ea754892b086a22474ba810c (127.24.71.193:38979)
W20260811 22:25:49.524926 30280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:49.525223 30280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:49.525264 30280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:49.529471 30280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:49.529546 30280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:49.529604 30280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:49.534613 30280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:49.535740 30280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:49.537307 30280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:49.547351 30286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:49.547353 30289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:49.547362 30287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:49.547582 30280 server_base.cc:1034] running on GCE node
I20260811 22:25:49.548230 30280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:49.548964 30280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:49.550176 30280 hybrid_clock.cc:648] HybridClock initialized: now 1786487149550132 us; error 59 us; skew 500 ppm
I20260811 22:25:49.552380 30280 webserver.cc:460] Webserver started at http://127.24.71.194:36263/ using document root <none> and password file <none>
I20260811 22:25:49.553224 30280 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:49.553329 30280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:49.553635 30280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:49.556646 30280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "e82d87c3c4c0421884bef9af9668f9ae"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "f1da7d0d6917926fb2810d409a990cdc"
server_key_iv: "4dcd726cba5022c6e7e0f2bc61ae4aa4"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.557245 30280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "e82d87c3c4c0421884bef9af9668f9ae"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "f1da7d0d6917926fb2810d409a990cdc"
server_key_iv: "4dcd726cba5022c6e7e0f2bc61ae4aa4"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.561250 30280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:49.564143 30295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.565774 30280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:49.565965 30280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "e82d87c3c4c0421884bef9af9668f9ae"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "f1da7d0d6917926fb2810d409a990cdc"
server_key_iv: "4dcd726cba5022c6e7e0f2bc61ae4aa4"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.566120 30280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:49.571097 30280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:49.571753 30280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:49.571983 30280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:49.572592 30280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:49.573549 30280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:49.573606 30280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.573674 30280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:49.573712 30280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.582896 30280 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:45359
I20260811 22:25:49.582924 30423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:45359 every 8 connection(s)
I20260811 22:25:49.583950 30280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:49.587817 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 30280
I20260811 22:25:49.588025 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:49.588338 24863 external_mini_cluster.cc:1428] Setting key dbf05727433db84598ab276ab0b326f6
I20260811 22:25:49.590641 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.585920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:47015 (local address 127.24.71.254:45885)
0811 22:25:49.586105 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.586110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.587168 (+ 1058us) server_negotiation.cc:407] Connection header received
0811 22:25:49.588697 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.588700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.588753 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.588846 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.590152 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.590155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.590158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.590170 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.590191 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.590199 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.590276 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.590494 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.590534 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:25:49.590675 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45885
--builtin_ntp_servers=127.24.71.212:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:25:49.591820 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.586306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:45885 (local address 127.24.71.194:47015)
0811 22:25:49.586997 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:49.587046 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:49.588455 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:49.589072 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:49.589079 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:49.589170 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:49.589833 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:49.589835 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:49.589901 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:49.589991 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:49.590293 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:49.590304 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:49.590312 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:25:49.590502 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:25:49.590787 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":132,"threads_started":1}
I20260811 22:25:49.593077 30424 heartbeater.cc:347] Connected to a master server at 127.24.71.254:45885
I20260811 22:25:49.593336 30424 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:49.593835 30424 heartbeater.cc:511] Master 127.24.71.254:45885 requested a full tablet report, sending...
I20260811 22:25:49.595043 30066 ts_manager.cc:194] Registered new tserver with Master: e82d87c3c4c0421884bef9af9668f9ae (127.24.71.194:45359)
W20260811 22:25:49.710438 30427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:49.710712 30427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:49.710753 30427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:49.714841 30427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:49.714908 30427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:49.714964 30427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:49.720527 30427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:49.721695 30427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:49.723094 30427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:49.734184 30435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:49.734118 30433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:49.734118 30432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:49.734454 30427 server_base.cc:1034] running on GCE node
I20260811 22:25:49.735011 30427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:49.735893 30427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:49.737121 30427 hybrid_clock.cc:648] HybridClock initialized: now 1786487149737103 us; error 32 us; skew 500 ppm
I20260811 22:25:49.739289 30427 webserver.cc:460] Webserver started at http://127.24.71.195:40311/ using document root <none> and password file <none>
I20260811 22:25:49.739969 30427 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:49.740074 30427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:49.740316 30427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:49.742501 30427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "81e000a4c84f4db196abe7aed1006c8a"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "b7ab85f380e8224450e365a15b8a8daf"
server_key_iv: "8e4c8bdbb2d9f1843c95d317c52ac57d"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.743145 30427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "81e000a4c84f4db196abe7aed1006c8a"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "b7ab85f380e8224450e365a15b8a8daf"
server_key_iv: "8e4c8bdbb2d9f1843c95d317c52ac57d"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.747056 30427 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:49.749781 30441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.751040 30427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:25:49.751180 30427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "81e000a4c84f4db196abe7aed1006c8a"
format_stamp: "Formatted at 2026-08-11 22:25:49 on dist-test-slave-mvvj"
server_key: "b7ab85f380e8224450e365a15b8a8daf"
server_key_iv: "8e4c8bdbb2d9f1843c95d317c52ac57d"
server_key_version: "encryptionkey@0"
I20260811 22:25:49.751303 30427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:49.756522 30427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:49.757264 30427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:49.757464 30427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:49.758052 30427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:49.759050 30427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:49.759099 30427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.759168 30427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:49.759202 30427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:49.769120 30427 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:36583
I20260811 22:25:49.769184 30566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:36583 every 8 connection(s)
I20260811 22:25:49.770205 30427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:49.777091 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.772298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:42865 (local address 127.24.71.254:45885)
0811 22:25:49.772469 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.772474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.773637 (+ 1163us) server_negotiation.cc:407] Connection header received
0811 22:25:49.775142 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.775146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.775198 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.775317 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.776582 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.776586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.776589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.776602 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.776626 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.776640 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.776739 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.776889 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.776947 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:25:49.778013 30569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.772710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:45885 (local address 127.24.71.195:42865)
0811 22:25:49.773416 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:49.773513 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:49.774900 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:49.775509 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:49.775517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:49.775618 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:49.776240 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:49.776242 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:49.776285 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:49.776359 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:49.776730 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:49.776739 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:49.776747 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:25:49.776938 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 22:25:49.777187 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":131,"threads_started":1}
I20260811 22:25:49.779055 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 30427
I20260811 22:25:49.779171 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:49.779328 30567 heartbeater.cc:347] Connected to a master server at 127.24.71.254:45885
I20260811 22:25:49.779587 24863 external_mini_cluster.cc:1428] Setting key 9d81afd9aac2086e7ac94f8b71a0a785
I20260811 22:25:49.779618 30567 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:49.780143 30567 heartbeater.cc:511] Master 127.24.71.254:45885 requested a full tablet report, sending...
I20260811 22:25:49.781324 30065 ts_manager.cc:194] Registered new tserver with Master: 81e000a4c84f4db196abe7aed1006c8a (127.24.71.195:36583)
I20260811 22:25:49.793679 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:25:49.797742 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.795950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.24.71.254:45885)
0811 22:25:49.796124 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.796128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.796261 (+ 133us) server_negotiation.cc:407] Connection header received
0811 22:25:49.796430 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.796433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.796484 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.796572 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.797253 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.797257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.797259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.797271 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.797295 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.797308 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.797399 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.797545 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.797589 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:25:49.801342 30065 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:49.804580 30065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:49.817163 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.813210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:38979 (local address 127.0.0.1:38522)
0811 22:25:49.814475 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:49.814489 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:49.814578 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:49.815782 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:49.815785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:49.815803 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:49.816301 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:49.816304 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:49.816317 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:49.816352 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:49.816768 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:49.816778 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:49.816784 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:25:49.816909 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:25:49.816984 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"thread_start_us":111,"threads_started":1}
I20260811 22:25:49.817199 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.813177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:36583 (local address 127.0.0.1:53270)
0811 22:25:49.814115 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:49.814169 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:49.814277 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:49.815041 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:49.815049 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:49.815130 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:49.815813 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:49.815815 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:49.815866 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:49.815966 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:49.816883 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:49.816890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:49.816898 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:25:49.816976 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:25:49.817021 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":82,"threads_started":1}
I20260811 22:25:49.817715 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.813432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.24.71.193:38979)
0811 22:25:49.813860 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.813874 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.814579 (+ 705us) server_negotiation.cc:407] Connection header received
0811 22:25:49.814665 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.814671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.814874 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.815040 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.816545 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.816552 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.816566 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.816580 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.816627 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.816668 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.816787 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.817473 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.817556 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":112,"threads_started":1}
I20260811 22:25:49.817788 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.814002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56210 (local address 127.24.71.194:45359)
0811 22:25:49.814810 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.814821 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.815047 (+ 226us) server_negotiation.cc:407] Connection header received
0811 22:25:49.815215 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.815221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.815391 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.815499 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.816737 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.816743 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.816755 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.816773 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.816822 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.816860 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.816976 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.817152 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.817629 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":133,"threads_started":1}
I20260811 22:25:49.817812 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.814079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.24.71.195:36583)
0811 22:25:49.814541 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.814554 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.814577 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:25:49.814633 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.814641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.814848 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.815010 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.816604 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.816610 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.816619 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.816630 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.816700 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.816733 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.817534 (+ 801us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.817589 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.817663 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":118,"threads_started":1}
I20260811 22:25:49.818181 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.813487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:45359 (local address 127.0.0.1:56210)
0811 22:25:49.814960 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:49.814973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:49.815104 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:49.815787 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:49.815792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:49.815808 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:49.816545 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:49.816547 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:49.816558 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:49.816585 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:49.817021 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:49.817025 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:49.817028 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:25:49.818015 (+ 987us) client_negotiation.cc:240] Negotiation successful
0811 22:25:49.818061 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1392,"thread_start_us":96,"threads_started":1}
I20260811 22:25:49.820799 30494 tablet_service.cc:1467] Processing CreateTablet for tablet 65d60d491e094c6497bfb423c456ca5c (DEFAULT_TABLE table=test-table [id=81640fd6df244a9da09b8f972a88f2d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:49.820798 30202 tablet_service.cc:1467] Processing CreateTablet for tablet 65d60d491e094c6497bfb423c456ca5c (DEFAULT_TABLE table=test-table [id=81640fd6df244a9da09b8f972a88f2d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:49.821175 30349 tablet_service.cc:1467] Processing CreateTablet for tablet 65d60d491e094c6497bfb423c456ca5c (DEFAULT_TABLE table=test-table [id=81640fd6df244a9da09b8f972a88f2d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:49.821794 30494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65d60d491e094c6497bfb423c456ca5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:49.821906 30202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65d60d491e094c6497bfb423c456ca5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:49.822122 30349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65d60d491e094c6497bfb423c456ca5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:49.828981 30586 tablet_bootstrap.cc:492] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: Bootstrap starting.
I20260811 22:25:49.830152 30588 tablet_bootstrap.cc:492] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: Bootstrap starting.
I20260811 22:25:49.830259 30587 tablet_bootstrap.cc:492] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c: Bootstrap starting.
I20260811 22:25:49.832434 30587 tablet_bootstrap.cc:654] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:49.832482 30586 tablet_bootstrap.cc:654] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:49.832481 30588 tablet_bootstrap.cc:654] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:49.833299 30587 log.cc:824] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:49.833346 30586 log.cc:824] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:49.833346 30588 log.cc:824] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:49.835500 30586 tablet_bootstrap.cc:492] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: No bootstrap required, opened a new log
I20260811 22:25:49.835529 30587 tablet_bootstrap.cc:492] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c: No bootstrap required, opened a new log
I20260811 22:25:49.835529 30588 tablet_bootstrap.cc:492] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: No bootstrap required, opened a new log
I20260811 22:25:49.835726 30586 ts_tablet_manager.cc:1397] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:25:49.835726 30587 ts_tablet_manager.cc:1397] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:25:49.835726 30588 ts_tablet_manager.cc:1397] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:25:49.839402 30586 raft_consensus.cc:348] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } }
I20260811 22:25:49.839708 30586 raft_consensus.cc:374] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:49.839764 30586 raft_consensus.cc:729] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e000a4c84f4db196abe7aed1006c8a, State: Initialized, Role: FOLLOWER
I20260811 22:25:49.840346 30586 consensus_queue.cc:260] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } }
I20260811 22:25:49.840436 30587 raft_consensus.cc:348] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } }
I20260811 22:25:49.840437 30588 raft_consensus.cc:348] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } }
I20260811 22:25:49.840703 30588 raft_consensus.cc:374] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:49.840703 30587 raft_consensus.cc:374] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:49.840763 30588 raft_consensus.cc:729] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82d87c3c4c0421884bef9af9668f9ae, State: Initialized, Role: FOLLOWER
I20260811 22:25:49.840763 30587 raft_consensus.cc:729] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f163f1ea754892b086a22474ba810c, State: Initialized, Role: FOLLOWER
I20260811 22:25:49.841243 30567 heartbeater.cc:503] Master 127.24.71.254:45885 was elected leader, sending a full tablet report...
I20260811 22:25:49.841235 30587 consensus_queue.cc:260] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } }
I20260811 22:25:49.841552 30586 ts_tablet_manager.cc:1428] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:25:49.842010 30277 heartbeater.cc:503] Master 127.24.71.254:45885 was elected leader, sending a full tablet report...
I20260811 22:25:49.842206 30587 ts_tablet_manager.cc:1428] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:25:49.842446 30588 consensus_queue.cc:260] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } }
I20260811 22:25:49.843109 30424 heartbeater.cc:503] Master 127.24.71.254:45885 was elected leader, sending a full tablet report...
I20260811 22:25:49.844401 30588 ts_tablet_manager.cc:1428] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
W20260811 22:25:49.898922 30278 tablet.cc:2369] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:49.946857 30592 raft_consensus.cc:482] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:49.947111 30592 raft_consensus.cc:504] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } }
I20260811 22:25:49.948422 30592 leader_election.cc:288] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f163f1ea754892b086a22474ba810c (127.24.71.193:38979), e82d87c3c4c0421884bef9af9668f9ae (127.24.71.194:45359)
I20260811 22:25:49.951076 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.948888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:50979 (local address 127.24.71.194:45359)
0811 22:25:49.949191 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.949196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.949213 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:25:49.949263 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.949265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.949306 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.949387 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.950526 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.950531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.950534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.950549 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.950578 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.950599 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.950716 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.950884 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.950933 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260811 22:25:49.951076 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.948912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:43753 (local address 127.24.71.193:38979)
0811 22:25:49.949058 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.949064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.949531 (+ 467us) server_negotiation.cc:407] Connection header received
0811 22:25:49.949588 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.949591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.949629 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.949789 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.950541 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.950545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.950547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.950567 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.950634 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.950649 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.950765 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.950886 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.950947 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:25:49.951103 30569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.948773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:45359 (local address 127.24.71.195:50979)
0811 22:25:49.948927 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:49.948945 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:49.949045 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:49.949496 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:49.949499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:49.949512 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:49.950191 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:49.950230 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:49.950247 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:49.950303 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:49.950754 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:49.950758 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:49.950763 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:25:49.950880 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:25:49.950965 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":44}
I20260811 22:25:49.951117 30595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.948773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:38979 (local address 127.24.71.195:43753)
0811 22:25:49.949383 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:49.949400 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:49.949499 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:49.949812 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:49.949814 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:49.949827 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:49.950321 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:25:49.950323 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:49.950336 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:25:49.950364 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:49.950754 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:49.950758 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:49.950763 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:25:49.950869 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:25:49.950953 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":195,"thread_start_us":165,"threads_started":1}
I20260811 22:25:49.951967 30371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d60d491e094c6497bfb423c456ca5c" candidate_uuid: "81e000a4c84f4db196abe7aed1006c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82d87c3c4c0421884bef9af9668f9ae" is_pre_election: true
I20260811 22:25:49.951977 30225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d60d491e094c6497bfb423c456ca5c" candidate_uuid: "81e000a4c84f4db196abe7aed1006c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f163f1ea754892b086a22474ba810c" is_pre_election: true
I20260811 22:25:49.952294 30371 raft_consensus.cc:2454] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81e000a4c84f4db196abe7aed1006c8a in term 0.
I20260811 22:25:49.952313 30225 raft_consensus.cc:2454] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81e000a4c84f4db196abe7aed1006c8a in term 0.
I20260811 22:25:49.952863 30443 leader_election.cc:302] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 81e000a4c84f4db196abe7aed1006c8a, 95f163f1ea754892b086a22474ba810c, e82d87c3c4c0421884bef9af9668f9ae; no voters:
I20260811 22:25:49.953258 30592 raft_consensus.cc:2781] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:49.953374 30592 raft_consensus.cc:482] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:49.953436 30592 raft_consensus.cc:3037] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:49.954547 30592 raft_consensus.cc:504] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } }
I20260811 22:25:49.955013 30592 leader_election.cc:288] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [CANDIDATE]: Term 1 election: Requested vote from peers 95f163f1ea754892b086a22474ba810c (127.24.71.193:38979), e82d87c3c4c0421884bef9af9668f9ae (127.24.71.194:45359)
I20260811 22:25:49.955385 30225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d60d491e094c6497bfb423c456ca5c" candidate_uuid: "81e000a4c84f4db196abe7aed1006c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f163f1ea754892b086a22474ba810c"
I20260811 22:25:49.955436 30371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65d60d491e094c6497bfb423c456ca5c" candidate_uuid: "81e000a4c84f4db196abe7aed1006c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82d87c3c4c0421884bef9af9668f9ae"
I20260811 22:25:49.955544 30225 raft_consensus.cc:3037] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:49.955572 30371 raft_consensus.cc:3037] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:49.956736 30225 raft_consensus.cc:2454] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81e000a4c84f4db196abe7aed1006c8a in term 1.
I20260811 22:25:49.956765 30371 raft_consensus.cc:2454] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81e000a4c84f4db196abe7aed1006c8a in term 1.
I20260811 22:25:49.957085 30445 leader_election.cc:302] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81e000a4c84f4db196abe7aed1006c8a, 95f163f1ea754892b086a22474ba810c; no voters:
I20260811 22:25:49.957289 30592 raft_consensus.cc:2781] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:49.957510 30592 raft_consensus.cc:686] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 1 LEADER]: Becoming Leader. State: Replica: 81e000a4c84f4db196abe7aed1006c8a, State: Running, Role: LEADER
I20260811 22:25:49.957821 30592 consensus_queue.cc:237] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } }
I20260811 22:25:49.961444 30063 catalog_manager.cc:5495] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 81e000a4c84f4db196abe7aed1006c8a (127.24.71.195). New cstate: current_term: 1 leader_uuid: "81e000a4c84f4db196abe7aed1006c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e000a4c84f4db196abe7aed1006c8a" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 } health_report { overall_health: UNKNOWN } } }
I20260811 22:25:49.976922 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:49.975092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.24.71.195:36583)
0811 22:25:49.975236 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:49.975240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:49.975258 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:25:49.975421 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:49.975424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:49.975526 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:49.975602 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:49.976465 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:49.976468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:49.976470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:49.976481 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:49.976502 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:49.976510 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:49.976597 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:49.976666 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 22:25:49.976718 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:25:49.983371 30225 raft_consensus.cc:1261] T 65d60d491e094c6497bfb423c456ca5c P 95f163f1ea754892b086a22474ba810c [term 1 FOLLOWER]: Refusing update from remote peer 81e000a4c84f4db196abe7aed1006c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:49.983371 30371 raft_consensus.cc:1261] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [term 1 FOLLOWER]: Refusing update from remote peer 81e000a4c84f4db196abe7aed1006c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:49.984148 30592 consensus_queue.cc:1035] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82d87c3c4c0421884bef9af9668f9ae" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:49.984488 30596 consensus_queue.cc:1035] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f163f1ea754892b086a22474ba810c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 38979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:49.991927 30600 mvcc.cc:205] Tried to move back new op lower bound from 7317451366324064256 to 7317451366229901312. Current Snapshot: MvccSnapshot[applied={T|T < 7317451366324064256}]
I20260811 22:25:50.001327 30063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:50.001543 30063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:50.004288 30063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2363d81d1a2b4a5099ad0ea396f14d21: Sending DeleteTablet for 3 replicas of tablet 65d60d491e094c6497bfb423c456ca5c
I20260811 22:25:50.005455 30494 tablet_service.cc:1514] Processing DeleteTablet for tablet 65d60d491e094c6497bfb423c456ca5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:49 UTC) from {username='slave'} at 127.0.0.1:53270
I20260811 22:25:50.005456 30202 tablet_service.cc:1514] Processing DeleteTablet for tablet 65d60d491e094c6497bfb423c456ca5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:49 UTC) from {username='slave'} at 127.0.0.1:38522
I20260811 22:25:50.005893 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 30135
I20260811 22:25:50.006573 30349 tablet_service.cc:1514] Processing DeleteTablet for tablet 65d60d491e094c6497bfb423c456ca5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:49 UTC) from {username='slave'} at 127.0.0.1:56210
I20260811 22:25:50.007401 30609 tablet_replica.cc:330] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: stopping tablet replica
I20260811 22:25:50.007802 30609 raft_consensus.cc:2229] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:25:50.008277 30609 raft_consensus.cc:2258] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:50.008708 30611 tablet_replica.cc:330] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: stopping tablet replica
I20260811 22:25:50.009096 30611 raft_consensus.cc:2229] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:50.009516 30611 raft_consensus.cc:2258] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:50.011199 30609 ts_tablet_manager.cc:1905] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:50.012918 30611 ts_tablet_manager.cc:1905] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:50.015318 30609 ts_tablet_manager.cc:1918] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:50.015452 30609 log.cc:1196] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/65d60d491e094c6497bfb423c456ca5c
I20260811 22:25:50.015856 30609 ts_tablet_manager.cc:1939] T 65d60d491e094c6497bfb423c456ca5c P 81e000a4c84f4db196abe7aed1006c8a: Deleting consensus metadata
I20260811 22:25:50.016404 30611 ts_tablet_manager.cc:1918] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:50.016587 30611 log.cc:1196] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/65d60d491e094c6497bfb423c456ca5c
I20260811 22:25:50.017156 30611 ts_tablet_manager.cc:1939] T 65d60d491e094c6497bfb423c456ca5c P e82d87c3c4c0421884bef9af9668f9ae: Deleting consensus metadata
I20260811 22:25:50.017472 30050 catalog_manager.cc:4841] TS 81e000a4c84f4db196abe7aed1006c8a (127.24.71.195:36583): tablet 65d60d491e094c6497bfb423c456ca5c (table test-table [id=81640fd6df244a9da09b8f972a88f2d3]) successfully deleted
I20260811 22:25:50.018632 30048 catalog_manager.cc:4841] TS e82d87c3c4c0421884bef9af9668f9ae (127.24.71.194:45359): tablet 65d60d491e094c6497bfb423c456ca5c (table test-table [id=81640fd6df244a9da09b8f972a88f2d3]) successfully deleted
W20260811 22:25:50.019519 30049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:38979 (error 108)
I20260811 22:25:50.019572 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 30280
I20260811 22:25:50.023610 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:50.023181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:38979 (local address 127.0.0.1:38524)
0811 22:25:50.023371 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:50.023485 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:38979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:25:50.023891 30049 catalog_manager.cc:4584] TS 95f163f1ea754892b086a22474ba810c (127.24.71.193:38979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65d60d491e094c6497bfb423c456ca5c: Network error: Client connection negotiation failed: client connection to 127.24.71.193:38979: connect: Connection refused (error 111)
I20260811 22:25:50.032230 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 30427
I20260811 22:25:50.040872 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 30032
2026-08-11T22:25:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1108 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 22:25:50.066298 24863 mini_oidc.cc:152] Starting JWKS server
I20260811 22:25:50.068187 24863 webserver.cc:460] Webserver started at https://127.0.0.1:35525/ using document root <none> and password file <none>
I20260811 22:25:50.068387 24863 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:25:50.068667 24863 webserver.cc:460] Webserver started at http://127.0.0.1:44055/ using document root <none> and password file <none>
2026-08-11T22:25:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:50Z Disabled control of system clock
I20260811 22:25:50.079668 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:37583
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:37549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:37583
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35525/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/testchainca.cert with env {}
W20260811 22:25:50.196673 30618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:50.196951 30618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:50.196993 30618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:50.201095 30618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:50.201164 30618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:50.201184 30618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:50.201201 30618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:50.201216 30618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:25:50.201232 30618 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:25:50.201246 30618 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35525/jwks.json
I20260811 22:25:50.206835 30618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:37549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:37583
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:37583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35525/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:50.208024 30618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:50.209064 30618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:50.218474 30624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:50.218474 30626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:50.218536 30618 server_base.cc:1034] running on GCE node
W20260811 22:25:50.218487 30623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:50.219319 30618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:50.220696 30618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:50.221889 30618 hybrid_clock.cc:648] HybridClock initialized: now 1786487150221851 us; error 51 us; skew 500 ppm
I20260811 22:25:50.224309 30618 webserver.cc:460] Webserver started at http://127.24.71.254:35253/ using document root <none> and password file <none>
I20260811 22:25:50.224947 30618 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:50.225018 30618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:50.225253 30618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:50.227463 30618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "739ff7881a774172800b118b0de2f5d4"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-mvvj"
server_key: "3e991a66ac049ddbbd9f0ff8bf066699"
server_key_iv: "b29b3c23d4432fb171451032046ad4e5"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.228118 30618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "739ff7881a774172800b118b0de2f5d4"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-mvvj"
server_key: "3e991a66ac049ddbbd9f0ff8bf066699"
server_key_iv: "b29b3c23d4432fb171451032046ad4e5"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.232276 30618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:25:50.234946 30633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:50.236426 30618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:50.236626 30618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "739ff7881a774172800b118b0de2f5d4"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-mvvj"
server_key: "3e991a66ac049ddbbd9f0ff8bf066699"
server_key_iv: "b29b3c23d4432fb171451032046ad4e5"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.236765 30618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:50.258185 30618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:50.259034 30618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:50.259261 30618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:50.267143 30618 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:37583
I20260811 22:25:50.267155 30689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:37583 every 8 connection(s)
I20260811 22:25:50.268177 30618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:50.272253 30690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:50.276782 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 30618
I20260811 22:25:50.276954 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:50.277251 24863 external_mini_cluster.cc:1428] Setting key 14b3304c862eb7f197b525d2952c4cb3
I20260811 22:25:50.279783 30690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4: Bootstrap starting.
I20260811 22:25:50.282779 30690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:50.283960 30690 log.cc:824] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:50.286900 30690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4: No bootstrap required, opened a new log
I20260811 22:25:50.288731 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:50.278915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34966 (local address 127.24.71.254:37583)
0811 22:25:50.279659 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:50.279678 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:50.279719 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:25:50.280590 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:50.280611 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:50.281093 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:50.281443 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:50.281850 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.283006 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.284501 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.285298 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.285467 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:50.286576 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:50.286582 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:50.286599 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:50.287113 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:50.287184 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:50.287209 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:50.287298 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:50.287566 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 22:25:50.287952 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":172,"threads_started":1}
I20260811 22:25:50.291790 30690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739ff7881a774172800b118b0de2f5d4" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37583 } }
I20260811 22:25:50.292152 30690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:50.292220 30690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739ff7881a774172800b118b0de2f5d4, State: Initialized, Role: FOLLOWER
I20260811 22:25:50.292424 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:50.292732 30690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739ff7881a774172800b118b0de2f5d4" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37583 } }
I20260811 22:25:50.292927 30690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:50.293026 30690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:50.293159 30690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:50.294478 30690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739ff7881a774172800b118b0de2f5d4" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37583 } }
I20260811 22:25:50.294917 30690 leader_election.cc:302] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 739ff7881a774172800b118b0de2f5d4; no voters:
I20260811 22:25:50.295275 30690 leader_election.cc:288] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:50.295482 30696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:50.295749 30696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [term 1 LEADER]: Becoming Leader. State: Replica: 739ff7881a774172800b118b0de2f5d4, State: Running, Role: LEADER
I20260811 22:25:50.296080 30696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739ff7881a774172800b118b0de2f5d4" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37583 } }
I20260811 22:25:50.296728 30690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:50.297899 30698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 739ff7881a774172800b118b0de2f5d4. Latest consensus state: current_term: 1 leader_uuid: "739ff7881a774172800b118b0de2f5d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739ff7881a774172800b118b0de2f5d4" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37583 } } }
I20260811 22:25:50.297904 30697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "739ff7881a774172800b118b0de2f5d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739ff7881a774172800b118b0de2f5d4" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37583 } } }
I20260811 22:25:50.298053 30697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:50.298053 30698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:50.298491 30708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:50.302004 30708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:50.308584 30708 catalog_manager.cc:1347] Generated new cluster ID: 4ac939d1d16649899654139f28ce3fb4
I20260811 22:25:50.308696 30708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:50.320719 30708 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:50.321875 30708 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:50.329792 30708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 739ff7881a774172800b118b0de2f5d4: Generated new TSK 0
I20260811 22:25:50.330546 30708 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:25:50.358052 30725 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 22:25:50.358276 30725 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:25:50.358989 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:50.341174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.24.71.254:37583)
0811 22:25:50.341308 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:50.341320 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:50.341590 (+ 270us) server_negotiation.cc:407] Connection header received
0811 22:25:50.341863 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:50.341869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:50.341927 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:50.342037 (+ 110us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:25:50.342310 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.342870 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.343752 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.344051 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.344167 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:50.344529 (+ 362us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:25:50.358470 (+ 13941us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35525/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:25:50.358815 (+ 345us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34970: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35525/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:25:50.359071 30693 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34970: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35525/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 22:25:50.360450 30724 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:50.341029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:37583 (local address 127.0.0.1:34970)
0811 22:25:50.341495 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:50.341509 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:50.341671 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:50.342055 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:50.342059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:50.342067 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:25:50.342194 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:50.342199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.342993 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.342997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:50.343622 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:50.343630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.344175 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.344179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:50.344282 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:50.344327 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:25:50.358743 (+ 14416us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35525/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:25:50.359021 (+ 278us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.71.254:37583: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35525/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":126,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.71.254:37583: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35525/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 22:25:50.361318 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 30618
2026-08-11T22:25:50Z chronyd exiting
I20260811 22:25:50.404680 24863 test_util.cc:175] -----------------------------------------------
I20260811 22:25:50.404773 24863 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786487121090521-24863-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (340 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 22:25:50.405917 24863 mini_oidc.cc:152] Starting JWKS server
I20260811 22:25:50.406821 24863 webserver.cc:460] Webserver started at https://127.0.0.1:42559/ using document root <none> and password file <none>
I20260811 22:25:50.406942 24863 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:25:50.407259 24863 webserver.cc:460] Webserver started at http://127.0.0.1:34377/ using document root <none> and password file <none>
2026-08-11T22:25:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:50Z Disabled control of system clock
I20260811 22:25:50.417469 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:41971
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:44343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:41971
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42559/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/testchainca.cert with env {}
W20260811 22:25:50.551563 30732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:50.551928 30732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:50.551988 30732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:50.557188 30732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:50.557271 30732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:50.557297 30732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:50.557322 30732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:50.557343 30732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:25:50.557374 30732 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:25:50.557410 30732 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42559/jwks.json
I20260811 22:25:50.563601 30732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:44343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:41971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:41971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42559/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:50.564743 30732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:50.565762 30732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:50.574946 30741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:50.574930 30738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:50.574959 30739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:50.575194 30732 server_base.cc:1034] running on GCE node
I20260811 22:25:50.576087 30732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:50.577378 30732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:50.578570 30732 hybrid_clock.cc:648] HybridClock initialized: now 1786487150578541 us; error 41 us; skew 500 ppm
I20260811 22:25:50.580837 30732 webserver.cc:460] Webserver started at http://127.24.71.254:39509/ using document root <none> and password file <none>
I20260811 22:25:50.581526 30732 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:50.581598 30732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:50.581859 30732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:50.584280 30732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "04e809a63eb64044a38a3a7c651422f8"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-mvvj"
server_key: "33d8d2540f2a3f3dc64d6ff34886ff1f"
server_key_iv: "0b3e172ef2f1c36fab0648780df9fb9e"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.584980 30732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "04e809a63eb64044a38a3a7c651422f8"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-mvvj"
server_key: "33d8d2540f2a3f3dc64d6ff34886ff1f"
server_key_iv: "0b3e172ef2f1c36fab0648780df9fb9e"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.589160 30732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:25:50.591989 30747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:50.593477 30732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:50.593647 30732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "04e809a63eb64044a38a3a7c651422f8"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-mvvj"
server_key: "33d8d2540f2a3f3dc64d6ff34886ff1f"
server_key_iv: "0b3e172ef2f1c36fab0648780df9fb9e"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.593775 30732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:50.620395 30732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:50.621232 30732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:50.621464 30732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:50.629398 30732 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:41971
I20260811 22:25:50.629462 30801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:41971 every 8 connection(s)
I20260811 22:25:50.630489 30732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:50.633826 30802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:50.634364 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 30732
I20260811 22:25:50.634462 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:50.634765 24863 external_mini_cluster.cc:1428] Setting key 19f2f87e25001517ec6745d962acd535
I20260811 22:25:50.641004 30802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8: Bootstrap starting.
I20260811 22:25:50.643887 30802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:50.644716 30802 log.cc:824] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:50.646103 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:50.636428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53592 (local address 127.24.71.254:41971)
0811 22:25:50.637174 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:50.637195 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:50.637243 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:25:50.638112 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:50.638136 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:50.638668 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:50.639050 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:50.639539 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.641008 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.642292 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.642827 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.642960 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:50.644021 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:50.644032 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:50.644064 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:50.644537 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:50.644597 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:50.644620 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:50.644706 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:50.644976 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 22:25:50.645281 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":111,"threads_started":1}
I20260811 22:25:50.646796 30802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8: No bootstrap required, opened a new log
I20260811 22:25:50.649381 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:50.651230 30802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e809a63eb64044a38a3a7c651422f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41971 } }
I20260811 22:25:50.651520 30802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:50.651582 30802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04e809a63eb64044a38a3a7c651422f8, State: Initialized, Role: FOLLOWER
I20260811 22:25:50.652045 30802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e809a63eb64044a38a3a7c651422f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41971 } }
I20260811 22:25:50.652199 30802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:50.652264 30802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:50.652364 30802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:50.653488 30802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e809a63eb64044a38a3a7c651422f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41971 } }
I20260811 22:25:50.653790 30802 leader_election.cc:302] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04e809a63eb64044a38a3a7c651422f8; no voters:
I20260811 22:25:50.654053 30802 leader_election.cc:288] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:50.654206 30807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:50.654500 30807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [term 1 LEADER]: Becoming Leader. State: Replica: 04e809a63eb64044a38a3a7c651422f8, State: Running, Role: LEADER
I20260811 22:25:50.654816 30807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e809a63eb64044a38a3a7c651422f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41971 } }
I20260811 22:25:50.655345 30802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:50.656761 30810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04e809a63eb64044a38a3a7c651422f8. Latest consensus state: current_term: 1 leader_uuid: "04e809a63eb64044a38a3a7c651422f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e809a63eb64044a38a3a7c651422f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41971 } } }
I20260811 22:25:50.656764 30809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04e809a63eb64044a38a3a7c651422f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e809a63eb64044a38a3a7c651422f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41971 } } }
I20260811 22:25:50.657038 30809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:50.657073 30810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:50.657438 30819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:50.660285 30819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:50.666455 30819 catalog_manager.cc:1347] Generated new cluster ID: d9983ced0dac4addb782483f2344bbbe
I20260811 22:25:50.666553 30819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:50.676724 30819 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:50.677788 30819 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:50.690873 30819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04e809a63eb64044a38a3a7c651422f8: Generated new TSK 0
I20260811 22:25:50.691736 30819 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:25:50.733882 30838 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 22:25:50.733992 30838 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:25:50.734695 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:50.710750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53604 (local address 127.24.71.254:41971)
0811 22:25:50.710929 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:50.710935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:50.711113 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:25:50.711278 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:50.711281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:50.711346 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:50.711466 (+ 120us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:25:50.711757 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.712506 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.713808 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.714083 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.714177 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:50.714571 (+ 394us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:25:50.734282 (+ 19711us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:25:50.734558 (+ 276us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53604: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 22:25:50.734797 30805 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53604: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 22:25:50.734928 30837 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:50.710663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:41971 (local address 127.0.0.1:53604)
0811 22:25:50.711003 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:50.711019 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:50.711143 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:50.711496 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:50.711500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:50.711508 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:25:50.711639 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:50.711646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.712647 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.712652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:50.713652 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:50.713663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.714215 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.714222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:50.714318 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:50.714350 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:25:50.734527 (+ 20177us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:25:50.734732 (+ 205us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.71.254:41971: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":116,"threads_started":1}
I20260811 22:25:50.735801 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 30732
2026-08-11T22:25:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (373 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 22:25:50.779017 24863 mini_oidc.cc:152] Starting JWKS server
I20260811 22:25:50.779950 24863 webserver.cc:460] Webserver started at https://127.0.0.1:40417/ using document root <none> and password file <none>
I20260811 22:25:50.780046 24863 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:25:50.780313 24863 webserver.cc:460] Webserver started at http://127.0.0.1:45177/ using document root <none> and password file <none>
2026-08-11T22:25:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:50Z Disabled control of system clock
I20260811 22:25:50.791035 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:36399
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:36567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:36399
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40417/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:25:50.909122 30844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:50.909461 30844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:50.909548 30844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:50.913600 30844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:50.913700 30844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:50.913730 30844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:50.913769 30844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:50.913805 30844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:25:50.913836 30844 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:25:50.913859 30844 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40417/jwks.json
I20260811 22:25:50.918921 30844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:36567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:36399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:36399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40417/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:50.920198 30844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:50.921674 30844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:50.931164 30850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:50.931164 30851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:50.931173 30853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:50.931613 30844 server_base.cc:1034] running on GCE node
I20260811 22:25:50.932315 30844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:50.933465 30844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:50.934644 30844 hybrid_clock.cc:648] HybridClock initialized: now 1786487150934622 us; error 34 us; skew 500 ppm
I20260811 22:25:50.936923 30844 webserver.cc:460] Webserver started at http://127.24.71.254:44105/ using document root <none> and password file <none>
I20260811 22:25:50.937557 30844 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:50.937624 30844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:50.937857 30844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:50.940301 30844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "53339d3d4d794dd2aceab7025b64e814"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-mvvj"
server_key: "7d87b1e4fa163d100fc7cfbc95f49b0d"
server_key_iv: "d5f1a3302cfb8f03f2a3433dc168de4a"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.940949 30844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "53339d3d4d794dd2aceab7025b64e814"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-mvvj"
server_key: "7d87b1e4fa163d100fc7cfbc95f49b0d"
server_key_iv: "d5f1a3302cfb8f03f2a3433dc168de4a"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.945531 30844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:50.948342 30859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:50.949738 30844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:50.949875 30844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "53339d3d4d794dd2aceab7025b64e814"
format_stamp: "Formatted at 2026-08-11 22:25:50 on dist-test-slave-mvvj"
server_key: "7d87b1e4fa163d100fc7cfbc95f49b0d"
server_key_iv: "d5f1a3302cfb8f03f2a3433dc168de4a"
server_key_version: "encryptionkey@0"
I20260811 22:25:50.950001 30844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:50.967360 30844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:50.968226 30844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:50.968446 30844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:50.975973 30844 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:36399
I20260811 22:25:50.975992 30914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:36399 every 8 connection(s)
I20260811 22:25:50.976966 30844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:50.978106 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 30844
I20260811 22:25:50.978227 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:50.978550 24863 external_mini_cluster.cc:1428] Setting key 57ad9bced03c173a25ede596bfdeb127
I20260811 22:25:50.980930 30915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:50.987633 30915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814: Bootstrap starting.
I20260811 22:25:50.988548 30918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:50.980083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.24.71.254:36399)
0811 22:25:50.980794 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:50.980813 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:50.980850 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:25:50.981597 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:50.981615 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:50.982021 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:50.982309 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:50.982647 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.983641 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.984951 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:50.985490 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:50.985626 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:50.986597 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:50.986611 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:50.986625 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:50.987023 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:50.987080 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:50.987101 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:50.987181 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:50.987350 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 22:25:50.987692 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":147,"threads_started":1}
I20260811 22:25:50.990314 30915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:50.991230 30915 log.cc:824] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:50.991910 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:50.993685 30915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814: No bootstrap required, opened a new log
I20260811 22:25:50.997629 30915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53339d3d4d794dd2aceab7025b64e814" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36399 } }
I20260811 22:25:50.997895 30915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:50.997989 30915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53339d3d4d794dd2aceab7025b64e814, State: Initialized, Role: FOLLOWER
I20260811 22:25:50.998478 30915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53339d3d4d794dd2aceab7025b64e814" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36399 } }
I20260811 22:25:50.998615 30915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:50.998715 30915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:50.998831 30915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:50.999918 30915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53339d3d4d794dd2aceab7025b64e814" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36399 } }
I20260811 22:25:51.000278 30915 leader_election.cc:302] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53339d3d4d794dd2aceab7025b64e814; no voters:
I20260811 22:25:51.000627 30915 leader_election.cc:288] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:51.000759 30920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:51.001068 30920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [term 1 LEADER]: Becoming Leader. State: Replica: 53339d3d4d794dd2aceab7025b64e814, State: Running, Role: LEADER
I20260811 22:25:51.001382 30920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53339d3d4d794dd2aceab7025b64e814" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36399 } }
I20260811 22:25:51.002049 30915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:51.003258 30922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53339d3d4d794dd2aceab7025b64e814. Latest consensus state: current_term: 1 leader_uuid: "53339d3d4d794dd2aceab7025b64e814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53339d3d4d794dd2aceab7025b64e814" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36399 } } }
I20260811 22:25:51.003402 30922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:51.003676 30921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53339d3d4d794dd2aceab7025b64e814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53339d3d4d794dd2aceab7025b64e814" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36399 } } }
I20260811 22:25:51.003772 30921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:51.003927 30929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:51.006788 30929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:51.013979 30929 catalog_manager.cc:1347] Generated new cluster ID: 0410beeee28149dfb0d8ee4be2bc7926
I20260811 22:25:51.014087 30929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:51.020186 30929 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:51.021176 30929 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:51.026705 30929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53339d3d4d794dd2aceab7025b64e814: Generated new TSK 0
I20260811 22:25:51.027542 30929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:51.042390 30918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:51.036771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40430 (local address 127.24.71.254:36399)
0811 22:25:51.036928 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:51.036933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:51.037234 (+ 301us) server_negotiation.cc:407] Connection header received
0811 22:25:51.037413 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:51.037416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:51.037514 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:51.037616 (+ 102us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:25:51.037898 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.038481 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.039761 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.040048 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.040134 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:51.040504 (+ 370us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:25:51.042013 (+ 1509us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:25:51.042246 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 22:25:51.042476 30918 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 22:25:51.044277 30945 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:51.036635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36399 (local address 127.0.0.1:40430)
0811 22:25:51.037114 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:51.037131 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:51.037262 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:51.037638 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:51.037642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:51.037650 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:25:51.037779 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:51.037786 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.038622 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.038626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:51.039630 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:51.039640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.040178 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.040182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:51.040276 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:51.040309 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:25:51.042184 (+ 1875us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:25:51.044094 (+ 1910us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.71.254:36399: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":1697,"thread_start_us":123,"threads_started":1}
I20260811 22:25:51.044902 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 30844
E20260811 22:25:51.055693 30946 webserver.cc:533] Webserver: sslize failed:
E20260811 22:25:51.055801 30946 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T22:25:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T22:25:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:51Z Disabled control of system clock
I20260811 22:25:51.099284 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:45577
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:34111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:45577
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/testchainca.cert with env {}
W20260811 22:25:51.214365 30950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:51.214674 30950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:51.214716 30950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:51.219096 30950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:51.219182 30950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:51.219198 30950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:51.219215 30950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:51.219231 30950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:25:51.219249 30950 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:25:51.219261 30950 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 22:25:51.225054 30950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:45577
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:45577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:51.226140 30950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:51.227139 30950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:51.236197 30956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:51.236327 30950 server_base.cc:1034] running on GCE node
W20260811 22:25:51.236255 30959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:51.236261 30955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:51.237156 30950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:51.238335 30950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:51.239516 30950 hybrid_clock.cc:648] HybridClock initialized: now 1786487151239498 us; error 34 us; skew 500 ppm
I20260811 22:25:51.241444 30950 webserver.cc:460] Webserver started at http://127.24.71.254:45341/ using document root <none> and password file <none>
I20260811 22:25:51.242007 30950 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:51.242069 30950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:51.242236 30950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:51.244488 30950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "ba6544dd19b9409f933380b936f97ff8"
format_stamp: "Formatted at 2026-08-11 22:25:51 on dist-test-slave-mvvj"
server_key: "12b16fb195cb5a40d193219f0d90af08"
server_key_iv: "71dcbc573a9759aed7f94641cef02738"
server_key_version: "encryptionkey@0"
I20260811 22:25:51.245113 30950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "ba6544dd19b9409f933380b936f97ff8"
format_stamp: "Formatted at 2026-08-11 22:25:51 on dist-test-slave-mvvj"
server_key: "12b16fb195cb5a40d193219f0d90af08"
server_key_iv: "71dcbc573a9759aed7f94641cef02738"
server_key_version: "encryptionkey@0"
I20260811 22:25:51.249046 30950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:25:51.251580 30966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:51.252952 30950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:51.253088 30950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "ba6544dd19b9409f933380b936f97ff8"
format_stamp: "Formatted at 2026-08-11 22:25:51 on dist-test-slave-mvvj"
server_key: "12b16fb195cb5a40d193219f0d90af08"
server_key_iv: "71dcbc573a9759aed7f94641cef02738"
server_key_version: "encryptionkey@0"
I20260811 22:25:51.253207 30950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:51.276505 30950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:51.277325 30950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:51.277556 30950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:51.285260 30950 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:45577
I20260811 22:25:51.285286 31020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:45577 every 8 connection(s)
I20260811 22:25:51.286312 30950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:51.289446 31021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:51.295207 31021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8: Bootstrap starting.
I20260811 22:25:51.296075 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 30950
I20260811 22:25:51.296224 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:51.296519 24863 external_mini_cluster.cc:1428] Setting key 389b459bbfe1706afbb90bb527ba8522
I20260811 22:25:51.298293 31021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:51.299074 31021 log.cc:824] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:51.301165 31021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8: No bootstrap required, opened a new log
I20260811 22:25:51.305699 31021 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6544dd19b9409f933380b936f97ff8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45577 } }
I20260811 22:25:51.305929 31021 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:51.305989 31021 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6544dd19b9409f933380b936f97ff8, State: Initialized, Role: FOLLOWER
I20260811 22:25:51.306308 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:51.297792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.24.71.254:45577)
0811 22:25:51.298525 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:51.298540 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:51.298576 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:25:51.299257 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:51.299276 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:51.299665 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:51.300058 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:51.300392 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.301435 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.302656 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.303202 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.303370 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:51.304329 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:51.304335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:51.304349 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:25:51.304758 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:51.304815 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:25:51.304835 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:51.304931 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:51.305099 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:25:51.305408 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":150,"threads_started":1}
I20260811 22:25:51.306445 31021 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6544dd19b9409f933380b936f97ff8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45577 } }
I20260811 22:25:51.306627 31021 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:51.306723 31021 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:51.306846 31021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:51.307813 31021 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6544dd19b9409f933380b936f97ff8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45577 } }
I20260811 22:25:51.308244 31021 leader_election.cc:302] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba6544dd19b9409f933380b936f97ff8; no voters:
I20260811 22:25:51.308626 31021 leader_election.cc:288] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:51.308722 31027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:51.308935 31027 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [term 1 LEADER]: Becoming Leader. State: Replica: ba6544dd19b9409f933380b936f97ff8, State: Running, Role: LEADER
I20260811 22:25:51.309243 31027 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6544dd19b9409f933380b936f97ff8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45577 } }
I20260811 22:25:51.309451 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:25:51.310063 31021 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:51.310986 31028 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba6544dd19b9409f933380b936f97ff8. Latest consensus state: current_term: 1 leader_uuid: "ba6544dd19b9409f933380b936f97ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6544dd19b9409f933380b936f97ff8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45577 } } }
I20260811 22:25:51.311124 31028 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:51.311450 31029 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba6544dd19b9409f933380b936f97ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6544dd19b9409f933380b936f97ff8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45577 } } }
I20260811 22:25:51.311563 31029 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:51.312148 31037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:51.315467 31037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:51.321058 31037 catalog_manager.cc:1347] Generated new cluster ID: 2f8f34fa4d63430e89cc959720480059
I20260811 22:25:51.321146 31037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:51.339797 31037 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:51.341094 31037 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:51.348434 31037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba6544dd19b9409f933380b936f97ff8: Generated new TSK 0
I20260811 22:25:51.349269 31037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:25:51.815776 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:51.360485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.24.71.254:45577)
0811 22:25:51.360668 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:51.360675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:51.360975 (+ 300us) server_negotiation.cc:407] Connection header received
0811 22:25:51.361139 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:51.361142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:51.361196 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:51.361300 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:25:51.361572 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.362163 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.363498 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.363780 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.364035 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:51.364394 (+ 359us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:25:51.815301 (+450907us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:25:51.815651 (+ 350us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41738: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 22:25:51.815912 31024 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41738: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 22:25:51.816143 31054 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:51.360475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:45577 (local address 127.0.0.1:41738)
0811 22:25:51.360846 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:51.360863 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:51.360996 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:51.361320 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:51.361323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:51.361331 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:25:51.361459 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:51.361466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.362285 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.362289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:51.363291 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:51.363302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:51.364066 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:51.364071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:51.364176 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:51.364208 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:25:51.815638 (+451430us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:25:51.815917 (+ 279us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.71.254:45577: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":167,"threads_started":1}
I20260811 22:25:51.816794 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 30950
2026-08-11T22:25:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (750 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (135 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (137 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:52 dist-test-slave-mvvj krb5kdc[31067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:52 dist-test-slave-mvvj krb5kdc[31067](info): set up 2 sockets
Aug 11 22:25:52 dist-test-slave-mvvj krb5kdc[31067](info): commencing operation
krb5kdc: starting...
W20260811 22:25:53.217571 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:53 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:25:53.359946 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:46073
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:46073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:53.480113 31082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:53.480401 31082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:53.480465 31082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:53.484589 31082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:53.484654 31082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:53.484674 31082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:53.484691 31082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:53.484735 31082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:53.490412 31082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:46073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:46073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:53.491617 31082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:53.492738 31082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:53.501969 31090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:53.501921 31088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:53.502003 31087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:53.502092 31082 server_base.cc:1034] running on GCE node
I20260811 22:25:53.502878 31082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:53.504156 31082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:53.505350 31082 hybrid_clock.cc:648] HybridClock initialized: now 1786487153505353 us; error 50 us; skew 500 ppm
Aug 11 22:25:53 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:53.511740 31082 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:25:53.512944 31082 webserver.cc:460] Webserver started at http://127.24.71.254:45733/ using document root <none> and password file <none>
I20260811 22:25:53.513568 31082 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:53.513667 31082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:53.513937 31082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:53.516283 31082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "2517d7b9bcd8441591c8c26e3a525ca1"
format_stamp: "Formatted at 2026-08-11 22:25:53 on dist-test-slave-mvvj"
server_key: "098332bf5ba97c6f1fbed203a3d33d42"
server_key_iv: "88f592fe119388eaafef4fb125a739e5"
server_key_version: "encryptionkey@0"
I20260811 22:25:53.516963 31082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "2517d7b9bcd8441591c8c26e3a525ca1"
format_stamp: "Formatted at 2026-08-11 22:25:53 on dist-test-slave-mvvj"
server_key: "098332bf5ba97c6f1fbed203a3d33d42"
server_key_iv: "88f592fe119388eaafef4fb125a739e5"
server_key_version: "encryptionkey@0"
I20260811 22:25:53.521211 31082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:53.523821 31097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:53.525216 31082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:53.525386 31082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "2517d7b9bcd8441591c8c26e3a525ca1"
format_stamp: "Formatted at 2026-08-11 22:25:53 on dist-test-slave-mvvj"
server_key: "098332bf5ba97c6f1fbed203a3d33d42"
server_key_iv: "88f592fe119388eaafef4fb125a739e5"
server_key_version: "encryptionkey@0"
I20260811 22:25:53.525513 31082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:53.541693 31082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:53.545194 31082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:53.545441 31082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:53.553583 31082 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:46073
I20260811 22:25:53.553642 31153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:46073 every 8 connection(s)
I20260811 22:25:53.554971 31082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:53.558243 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 31082
I20260811 22:25:53.558334 31154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:53.558422 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:53.558717 24863 external_mini_cluster.cc:1428] Setting key 23a91895718356453594f82989f91768
I20260811 22:25:53.567188 31154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1: Bootstrap starting.
Aug 11 22:25:53 dist-test-slave-mvvj krb5kdc[31067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:53.570843 31154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:53.572033 31154 log.cc:824] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:53.574903 31154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1: No bootstrap required, opened a new log
I20260811 22:25:53.576833 31157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:53.560371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37168 (local address 127.24.71.254:46073)
0811 22:25:53.560886 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:53.560935 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:53.560990 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:25:53.561894 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:53.561933 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:53.562445 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:53.562887 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:53.563673 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.565394 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.566347 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.567115 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.567309 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:53.571008 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:53.571015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:53.571035 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:53.571075 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:53.573360 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:53.573834 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:53.573844 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:53.573853 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:53.574021 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:53.574325 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:53.574328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:53.574330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:53.574906 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:53.575131 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:53.575422 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 22:25:53.575755 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":138,"threads_started":1}
I20260811 22:25:53.580140 31154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517d7b9bcd8441591c8c26e3a525ca1" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46073 } }
I20260811 22:25:53.580394 31154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:53.580458 31154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2517d7b9bcd8441591c8c26e3a525ca1, State: Initialized, Role: FOLLOWER
I20260811 22:25:53.580909 31154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517d7b9bcd8441591c8c26e3a525ca1" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46073 } }
I20260811 22:25:53.581043 31154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:53.581095 31154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:53.581174 31154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:53.582242 31154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517d7b9bcd8441591c8c26e3a525ca1" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46073 } }
I20260811 22:25:53.582690 31154 leader_election.cc:302] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2517d7b9bcd8441591c8c26e3a525ca1; no voters:
I20260811 22:25:53.583055 31154 leader_election.cc:288] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:53.583225 31160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:53.583465 31160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 2517d7b9bcd8441591c8c26e3a525ca1, State: Running, Role: LEADER
I20260811 22:25:53.583801 31160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517d7b9bcd8441591c8c26e3a525ca1" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46073 } }
I20260811 22:25:53.584525 31154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:53.585692 31161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2517d7b9bcd8441591c8c26e3a525ca1. Latest consensus state: current_term: 1 leader_uuid: "2517d7b9bcd8441591c8c26e3a525ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517d7b9bcd8441591c8c26e3a525ca1" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46073 } } }
I20260811 22:25:53.585692 31162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2517d7b9bcd8441591c8c26e3a525ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517d7b9bcd8441591c8c26e3a525ca1" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46073 } } }
I20260811 22:25:53.585894 31162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:53.585893 31161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:53.586988 31170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:53.590044 31170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:53.596262 31170 catalog_manager.cc:1347] Generated new cluster ID: b3116264ed4a46b1a6bae60dcc088625
I20260811 22:25:53.596377 31170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:53.603694 31170 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:53.604779 31170 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:53.616142 31170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2517d7b9bcd8441591c8c26e3a525ca1: Generated new TSK 0
I20260811 22:25:53.616928 31170 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:25:53.655480 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46073
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:53.776153 31182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:53.776443 31182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:53.776494 31182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:53.780848 31182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:53.780925 31182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:53.781016 31182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:53.786870 31182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:53.788120 31182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:53.789194 31182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:53.799755 31191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:53.799752 31188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:53.799754 31189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:53.800205 31182 server_base.cc:1034] running on GCE node
I20260811 22:25:53.800662 31182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:53.801339 31182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:53.802589 31182 hybrid_clock.cc:648] HybridClock initialized: now 1786487153802547 us; error 54 us; skew 500 ppm
Aug 11 22:25:53 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:53.805641 31182 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:25:53.806663 31182 webserver.cc:460] Webserver started at http://127.24.71.193:41179/ using document root <none> and password file <none>
I20260811 22:25:53.807266 31182 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:53.807390 31182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:53.807659 31182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:53.809973 31182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "1b3a2784b75f447fa52b6d6c3480185d"
format_stamp: "Formatted at 2026-08-11 22:25:53 on dist-test-slave-mvvj"
server_key: "022da90ae9f1c32630a6c995f424e441"
server_key_iv: "9bb71986b27369529d1be5b3223ab14d"
server_key_version: "encryptionkey@0"
I20260811 22:25:53.810691 31182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "1b3a2784b75f447fa52b6d6c3480185d"
format_stamp: "Formatted at 2026-08-11 22:25:53 on dist-test-slave-mvvj"
server_key: "022da90ae9f1c32630a6c995f424e441"
server_key_iv: "9bb71986b27369529d1be5b3223ab14d"
server_key_version: "encryptionkey@0"
I20260811 22:25:53.814894 31182 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:53.817646 31198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:53.819022 31182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:53.819180 31182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "1b3a2784b75f447fa52b6d6c3480185d"
format_stamp: "Formatted at 2026-08-11 22:25:53 on dist-test-slave-mvvj"
server_key: "022da90ae9f1c32630a6c995f424e441"
server_key_iv: "9bb71986b27369529d1be5b3223ab14d"
server_key_version: "encryptionkey@0"
I20260811 22:25:53.819307 31182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:53.840829 31182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:53.844570 31182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:53.844797 31182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:53.845453 31182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:53.846340 31182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:53.846416 31182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:53.846530 31182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:53.846577 31182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:53.856648 31182 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:40217
I20260811 22:25:53.856665 31321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:40217 every 8 connection(s)
I20260811 22:25:53.857671 31182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:53.863754 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 31182
I20260811 22:25:53.863969 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:53.864321 24863 external_mini_cluster.cc:1428] Setting key 28078320c3dbe90c1a8ce3bfde0ece6b
Aug 11 22:25:53 dist-test-slave-mvvj krb5kdc[31067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487153, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:53.873787 31157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:53.859905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:54663 (local address 127.24.71.254:46073)
0811 22:25:53.860076 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:53.860082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:53.861073 (+ 991us) server_negotiation.cc:407] Connection header received
0811 22:25:53.862525 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:53.862528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:53.862581 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:53.862671 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:53.864030 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.864810 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.866189 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.866696 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.866835 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:53.869619 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:53.869622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:53.869625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:53.869655 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:53.871245 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:53.871970 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:53.871973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:53.871974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:53.872065 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:53.872507 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:53.872510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:53.872511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:53.872710 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:53.872792 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:53.873533 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 22:25:53.873654 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:25:53.875047 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:53.860370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:46073 (local address 127.24.71.193:54663)
0811 22:25:53.860901 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:53.860950 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:53.862270 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:53.862852 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:53.862859 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:53.863270 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:53.863798 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:53.863848 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.864964 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.864967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:53.866012 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:53.866028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:53.866862 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:53.866865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:53.867013 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:53.867791 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:53.867810 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:53.869408 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:53.871384 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:53.871394 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:53.871415 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:53.871862 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:53.872168 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:53.872171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:53.872174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:53.872387 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:53.872830 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:53.872840 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:53.873231 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 22:25:53.873478 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 22:25:53.873892 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":129,"threads_started":1}
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:25:53.876550 31322 heartbeater.cc:347] Connected to a master server at 127.24.71.254:46073
I20260811 22:25:53.876778 31322 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:53.877256 31322 heartbeater.cc:511] Master 127.24.71.254:46073 requested a full tablet report, sending...
I20260811 22:25:53.879042 31115 ts_manager.cc:194] Registered new tserver with Master: 1b3a2784b75f447fa52b6d6c3480185d (127.24.71.193:40217)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
I20260811 22:25:53.880476 31115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:54663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:25:53.898695 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46073
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:54.019146 31327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:54.019556 31327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:54.019646 31327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:54.024860 31327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:54.024974 31327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:54.025072 31327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:54.031711 31327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:54.033079 31327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:54.034269 31327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:54.045342 31333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:54.045406 31334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:54.045626 31327 server_base.cc:1034] running on GCE node
W20260811 22:25:54.045342 31336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:54.046233 31327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:54.046974 31327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:54.048179 31327 hybrid_clock.cc:648] HybridClock initialized: now 1786487154048157 us; error 34 us; skew 500 ppm
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:54.051584 31327 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:25:54.052683 31327 webserver.cc:460] Webserver started at http://127.24.71.194:38051/ using document root <none> and password file <none>
I20260811 22:25:54.053330 31327 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:54.053439 31327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:54.053699 31327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:54.056074 31327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "23af7749fdb14d4281249590e09fc280"
format_stamp: "Formatted at 2026-08-11 22:25:54 on dist-test-slave-mvvj"
server_key: "2675cdbdb4e1dd9cc85c7b1de8e6d049"
server_key_iv: "f7995aa5cdb91c5acdd2fe05130bfc2b"
server_key_version: "encryptionkey@0"
I20260811 22:25:54.056701 31327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "23af7749fdb14d4281249590e09fc280"
format_stamp: "Formatted at 2026-08-11 22:25:54 on dist-test-slave-mvvj"
server_key: "2675cdbdb4e1dd9cc85c7b1de8e6d049"
server_key_iv: "f7995aa5cdb91c5acdd2fe05130bfc2b"
server_key_version: "encryptionkey@0"
I20260811 22:25:54.061085 31327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:25:54.063956 31343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:54.065330 31327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:54.065510 31327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "23af7749fdb14d4281249590e09fc280"
format_stamp: "Formatted at 2026-08-11 22:25:54 on dist-test-slave-mvvj"
server_key: "2675cdbdb4e1dd9cc85c7b1de8e6d049"
server_key_iv: "f7995aa5cdb91c5acdd2fe05130bfc2b"
server_key_version: "encryptionkey@0"
I20260811 22:25:54.065644 31327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:54.080586 31327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:54.084079 31327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:54.084300 31327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:54.085068 31327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:54.086055 31327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:54.086107 31327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:54.086179 31327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:54.086212 31327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:54.095444 31327 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:45083
I20260811 22:25:54.095467 31466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:45083 every 8 connection(s)
I20260811 22:25:54.096539 31327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:54.106799 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 31327
I20260811 22:25:54.106973 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:54.107344 24863 external_mini_cluster.cc:1428] Setting key 0c5fe7979ecbf7b6e2765137c2ccfa63
I20260811 22:25:54.112062 31157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:54.098697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:42723 (local address 127.24.71.254:46073)
0811 22:25:54.098851 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:54.098855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:54.099731 (+ 876us) server_negotiation.cc:407] Connection header received
0811 22:25:54.101170 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:54.101173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:54.101233 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:54.101343 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:54.102539 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.103084 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.104333 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.104567 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.104684 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:54.107573 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:54.107576 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:54.107579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:54.107606 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:54.109411 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.110094 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.110097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.110098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.110182 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.110604 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.110606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.110608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.110787 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:54.110866 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:54.111620 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 22:25:54.111775 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:25:54.112870 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:54.099041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:46073 (local address 127.24.71.194:42723)
0811 22:25:54.099571 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:54.099617 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:54.100955 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:54.101525 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:54.101532 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:54.101905 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:54.102385 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:54.102397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.103217 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.103225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:54.104209 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:54.104219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.104740 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.104745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:54.104921 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:54.105550 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:54.105567 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:54.107245 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:54.109533 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:54.109539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:54.109557 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:54.109970 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:54.110274 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:54.110280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:54.110285 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:54.110501 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:54.110896 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:54.110905 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:54.111280 (+ 375us) client_negotiation.cc:769] Sending connection context
0811 22:25:54.111543 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 22:25:54.111881 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":116,"threads_started":1}
I20260811 22:25:54.114252 31467 heartbeater.cc:347] Connected to a master server at 127.24.71.254:46073
I20260811 22:25:54.114578 31467 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:54.115164 31467 heartbeater.cc:511] Master 127.24.71.254:46073 requested a full tablet report, sending...
I20260811 22:25:54.116433 31115 ts_manager.cc:194] Registered new tserver with Master: 23af7749fdb14d4281249590e09fc280 (127.24.71.194:45083)
I20260811 22:25:54.117127 31115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:42723
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:25:54.143236 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46073
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:54.261384 31472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:54.261642 31472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:54.261718 31472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:54.265729 31472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:54.265786 31472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:54.265897 31472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:54.271241 31472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:54.272339 31472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:54.273439 31472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:54.283437 31477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:54.283504 31472 server_base.cc:1034] running on GCE node
W20260811 22:25:54.283447 31478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:54.283483 31480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:54.284269 31472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:54.285076 31472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:54.286278 31472 hybrid_clock.cc:648] HybridClock initialized: now 1786487154286225 us; error 60 us; skew 500 ppm
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:54.289697 31472 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:25:54.290995 31472 webserver.cc:460] Webserver started at http://127.24.71.195:35319/ using document root <none> and password file <none>
I20260811 22:25:54.291669 31472 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:54.291740 31472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:54.292028 31472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:54.294358 31472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "495fda3fa79b4f03ad04f2704326f1ff"
format_stamp: "Formatted at 2026-08-11 22:25:54 on dist-test-slave-mvvj"
server_key: "f486ac46c14c747a734e1ebcb9e6bdfb"
server_key_iv: "43d1459a0475f69b619f2737764863bf"
server_key_version: "encryptionkey@0"
I20260811 22:25:54.294988 31472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "495fda3fa79b4f03ad04f2704326f1ff"
format_stamp: "Formatted at 2026-08-11 22:25:54 on dist-test-slave-mvvj"
server_key: "f486ac46c14c747a734e1ebcb9e6bdfb"
server_key_iv: "43d1459a0475f69b619f2737764863bf"
server_key_version: "encryptionkey@0"
I20260811 22:25:54.299331 31472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 22:25:54.302131 31489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:54.303504 31472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:25:54.303711 31472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "495fda3fa79b4f03ad04f2704326f1ff"
format_stamp: "Formatted at 2026-08-11 22:25:54 on dist-test-slave-mvvj"
server_key: "f486ac46c14c747a734e1ebcb9e6bdfb"
server_key_iv: "43d1459a0475f69b619f2737764863bf"
server_key_version: "encryptionkey@0"
I20260811 22:25:54.303879 31472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:54.328089 31472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:54.331482 31472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:54.331712 31472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:54.332403 31472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:54.333325 31472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:54.333401 31472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:54.333477 31472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:54.333527 31472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:54.342784 31472 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:45375
I20260811 22:25:54.342815 31617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:45375 every 8 connection(s)
I20260811 22:25:54.343716 31472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:54.351688 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 31472
I20260811 22:25:54.351862 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:54.352296 24863 external_mini_cluster.cc:1428] Setting key deac866ceb665e505964349693cc97d1
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:25:54.358717 31157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:54.345680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:52175 (local address 127.24.71.254:46073)
0811 22:25:54.345861 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:54.345866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:54.346731 (+ 865us) server_negotiation.cc:407] Connection header received
0811 22:25:54.348164 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:54.348167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:54.348217 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:54.348295 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:54.349490 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.350039 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.351196 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.351433 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.351535 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:54.354884 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:54.354888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:54.354891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:54.354923 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:54.356456 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.357028 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.357031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.357032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.357111 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.357435 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.357438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.357439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.357676 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:54.357761 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:54.358407 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 22:25:54.358527 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:25:54.359769 31621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:54.346074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:46073 (local address 127.24.71.195:52175)
0811 22:25:54.346570 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:54.346616 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:54.347938 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:54.348467 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:54.348474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:54.348822 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:54.349341 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:54.349355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.350153 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.350156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:54.351053 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:54.351063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.351633 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.351638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:54.351899 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:54.352682 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:54.352709 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:54.354710 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:54.356611 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:54.356617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:54.356643 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:54.356931 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:54.357196 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:54.357199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:54.357200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:54.357344 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:54.357810 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:54.357816 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:54.358112 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 22:25:54.358307 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 22:25:54.358609 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":124,"threads_started":1}
I20260811 22:25:54.361116 31619 heartbeater.cc:347] Connected to a master server at 127.24.71.254:46073
I20260811 22:25:54.361419 31619 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:54.361979 31619 heartbeater.cc:511] Master 127.24.71.254:46073 requested a full tablet report, sending...
I20260811 22:25:54.363107 31115 ts_manager.cc:194] Registered new tserver with Master: 495fda3fa79b4f03ad04f2704326f1ff (127.24.71.195:45375)
I20260811 22:25:54.363715 31115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:52175
I20260811 22:25:54.366634 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/test-admin.keytab.
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:54.386111 24863 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 22:25:54.389361 31626 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:54.387656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56404 (local address 127.24.71.193:40217)
0811 22:25:54.388067 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:54.388074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:54.388102 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:25:54.388165 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:54.388170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:54.388370 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:54.388508 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:54.389192 (+ 684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56404: BlockingRecv error: recv got EOF from 127.0.0.1:56404 (error 108)
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":122,"threads_started":1}
W20260811 22:25:54.389369 24863 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:54.390542 24863 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:54.391640 24863 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:25:54.402865 31626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:54.392747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.24.71.193:40217)
0811 22:25:54.392923 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:54.392927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:54.392990 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:25:54.393141 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:54.393145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:54.393196 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:54.393279 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:54.393816 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.394667 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.395615 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.396141 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.396240 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:54.399098 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:54.399110 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:54.399127 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:54.399154 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:54.400955 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.401454 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.401460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.401461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.401550 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.401852 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.401855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.401856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.402123 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:54.402293 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:54.402510 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 22:25:54.402700 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 22:25:54.407101 31626 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:54.406092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56434 (local address 127.24.71.193:40217)
0811 22:25:54.406248 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:54.406251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:54.406266 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:25:54.406382 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:54.406385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:54.406436 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:54.406510 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:54.406987 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56434: BlockingRecv error: recv got EOF from 127.0.0.1:56434 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 22:25:54.407181 24863 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:54.408699 24863 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:54.409829 24863 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:25:54.419736 31626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:54.410839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56438 (local address 127.24.71.193:40217)
0811 22:25:54.410984 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:54.410988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:54.411003 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:25:54.411099 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:54.411102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:54.411144 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:54.411217 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:54.411779 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.412389 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.413236 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.413493 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.413587 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:54.416136 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:54.416139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:54.416142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:54.416169 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:54.417966 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.418442 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.418445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.418447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.418536 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.418837 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.418840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.418842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.419041 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:54.419179 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:54.419367 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 22:25:54.419518 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 22:25:54.422750 31626 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:25:54.421773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56442 (local address 127.24.71.193:40217)
0811 22:25:54.421933 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:54.421937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:54.421950 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:25:54.422123 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:54.422125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:54.422166 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:54.422233 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:54.422645 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56442: BlockingRecv error: recv got EOF from 127.0.0.1:56442 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 22:25:54.423312 24863 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786487121090521-24863-0/krb5kdc/krb5cc)
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:54.424459 24863 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:54.425380 24863 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:25:54.434692 31626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:54.426236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.24.71.193:40217)
0811 22:25:54.426374 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:54.426377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:54.426390 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:25:54.426548 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:54.426550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:54.426589 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:54.426659 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:54.427206 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.427709 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.428566 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:54.428802 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:54.428883 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:54.431264 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:54.431267 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:54.431269 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:54.431293 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:54.432977 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.433436 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.433440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.433441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.433531 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:54.433807 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:54.433811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:54.433813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:54.433998 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:54.434140 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:54.434380 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 22:25:54.434539 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:25:54.436061 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 31182
I20260811 22:25:54.446808 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 31327
I20260811 22:25:54.455094 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 31472
I20260811 22:25:54.464159 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 31082
2026-08-11T22:25:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2378 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31640](info): set up 2 sockets
Aug 11 22:25:54 dist-test-slave-mvvj krb5kdc[31640](info): commencing operation
krb5kdc: starting...
W20260811 22:25:55.611196 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:25:55 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:25:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:25:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.254.keytab.
Entry for principal oryx/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.254.keytab.
I20260811 22:25:55.760288 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:35027
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.254.keytab
--principal=oryx/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:35027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:55.884776 31655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:55.885207 31655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:55.885303 31655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:55.889518 31655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:55.889583 31655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:55.889603 31655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:55.889621 31655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:55.889636 31655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:55.895684 31655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:35027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.254.keytab
--principal=oryx/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:35027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:55.896857 31655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:55.897956 31655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:55.907155 31661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:55.907125 31662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:55.907245 31655 server_base.cc:1034] running on GCE node
W20260811 22:25:55.907214 31664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:55.908188 31655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:55.909377 31655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:55.910621 31655 hybrid_clock.cc:648] HybridClock initialized: now 1786487155910589 us; error 48 us; skew 500 ppm
Aug 11 22:25:55 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487155, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:55.913759 31655 init.cc:377] Logged in from keytab as oryx/127.24.71.254@KRBTEST.COM (short username oryx)
I20260811 22:25:55.914822 31655 webserver.cc:460] Webserver started at http://127.24.71.254:42799/ using document root <none> and password file <none>
I20260811 22:25:55.915469 31655 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:55.915572 31655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:55.915937 31655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:55.918399 31655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "9918892880914ddf8b203bc656c08730"
format_stamp: "Formatted at 2026-08-11 22:25:55 on dist-test-slave-mvvj"
server_key: "39ae48781fbd813f6f10d1777991b910"
server_key_iv: "2a5ebebb7eb0e36c14300fc966b0b2fc"
server_key_version: "encryptionkey@0"
I20260811 22:25:55.919059 31655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "9918892880914ddf8b203bc656c08730"
format_stamp: "Formatted at 2026-08-11 22:25:55 on dist-test-slave-mvvj"
server_key: "39ae48781fbd813f6f10d1777991b910"
server_key_iv: "2a5ebebb7eb0e36c14300fc966b0b2fc"
server_key_version: "encryptionkey@0"
I20260811 22:25:55.923105 31655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 22:25:55.925773 31671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:55.927043 31655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:25:55.927196 31655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "9918892880914ddf8b203bc656c08730"
format_stamp: "Formatted at 2026-08-11 22:25:55 on dist-test-slave-mvvj"
server_key: "39ae48781fbd813f6f10d1777991b910"
server_key_iv: "2a5ebebb7eb0e36c14300fc966b0b2fc"
server_key_version: "encryptionkey@0"
I20260811 22:25:55.927318 31655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:55.954895 31655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:55.958662 31655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:55.958890 31655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:55.967023 31655 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:35027
I20260811 22:25:55.967316 31727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:35027 every 8 connection(s)
I20260811 22:25:55.968205 31655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:25:55.968896 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 31655
I20260811 22:25:55.969061 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:25:55.969394 24863 external_mini_cluster.cc:1428] Setting key 138462523597ab15453afb5d53bb933a
I20260811 22:25:55.971807 31728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:55.977829 31728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: Bootstrap starting.
Aug 11 22:25:55 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:25:55.980697 31728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:55.981741 31728 log.cc:824] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:55.984411 31728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: No bootstrap required, opened a new log
I20260811 22:25:55.985402 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:55.970911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59204 (local address 127.24.71.254:35027)
0811 22:25:55.971421 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:55.971438 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:55.971478 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:25:55.972289 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:55.972309 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:55.972675 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:55.973086 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:55.973936 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:55.974967 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:55.975872 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:55.976411 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:55.976641 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:55.979631 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:55.979636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:55.979652 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:55.979685 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:55.981922 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:55.982486 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:55.982492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:55.982499 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:55.982626 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:55.982950 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:55.982954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:55.982955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:55.983536 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:55.983742 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:55.984089 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 22:25:55.984406 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":171,"threads_started":1}
I20260811 22:25:55.988343 31728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } }
I20260811 22:25:55.988590 31728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:55.988629 31728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9918892880914ddf8b203bc656c08730, State: Initialized, Role: FOLLOWER
I20260811 22:25:55.989017 31728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } }
I20260811 22:25:55.989133 31728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:25:55.989179 31728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:25:55.989249 31728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:55.990216 31728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } }
I20260811 22:25:55.990525 31728 leader_election.cc:302] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9918892880914ddf8b203bc656c08730; no voters:
I20260811 22:25:55.990798 31728 leader_election.cc:288] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:25:55.990980 31733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:55.991245 31733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 1 LEADER]: Becoming Leader. State: Replica: 9918892880914ddf8b203bc656c08730, State: Running, Role: LEADER
I20260811 22:25:55.991559 31733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } }
I20260811 22:25:55.992228 31728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:55.993335 31734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } }
I20260811 22:25:55.993499 31734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:55.993810 31735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9918892880914ddf8b203bc656c08730. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } }
I20260811 22:25:55.993906 31735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:55.994102 31742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:25:55.997485 31742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:25:56.004223 31742 catalog_manager.cc:1347] Generated new cluster ID: c8e20ee46ee141e7bac9731a76061819
I20260811 22:25:56.004359 31742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:25:56.012768 31742 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:25:56.013885 31742 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:25:56.022092 31742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: Generated new TSK 0
I20260811 22:25:56.022934 31742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab.
Entry for principal oryx/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab.
I20260811 22:25:56.064190 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:35027
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab
--principal=oryx/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:56.187595 31756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:56.187981 31756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:56.188033 31756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:56.192247 31756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:56.192309 31756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:56.192369 31756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:56.198130 31756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab
--principal=oryx/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:35027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:56.199275 31756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:56.200434 31756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:56.211757 31756 server_base.cc:1034] running on GCE node
W20260811 22:25:56.211691 31762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:56.211701 31761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:56.211694 31764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:56.212417 31756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:56.213161 31756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:56.214404 31756 hybrid_clock.cc:648] HybridClock initialized: now 1786487156214408 us; error 50 us; skew 500 ppm
Aug 11 22:25:56 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:56.217769 31756 init.cc:377] Logged in from keytab as oryx/127.24.71.193@KRBTEST.COM (short username oryx)
I20260811 22:25:56.218756 31756 webserver.cc:460] Webserver started at http://127.24.71.193:45731/ using document root <none> and password file <none>
I20260811 22:25:56.219329 31756 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:56.219414 31756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:56.219600 31756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:56.221844 31756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "35f107625eac4a55b7eb0301a96ff278"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "4bd6e833eae38b0ba8b77f72fc7ecbb0"
server_key_iv: "192e9a34f5392eefd5ac3ccd87fa9f20"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.222417 31756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "35f107625eac4a55b7eb0301a96ff278"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "4bd6e833eae38b0ba8b77f72fc7ecbb0"
server_key_iv: "192e9a34f5392eefd5ac3ccd87fa9f20"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.226651 31756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:25:56.229591 31771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.231348 31756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:25:56.231509 31756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "35f107625eac4a55b7eb0301a96ff278"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "4bd6e833eae38b0ba8b77f72fc7ecbb0"
server_key_iv: "192e9a34f5392eefd5ac3ccd87fa9f20"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.231652 31756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:56.257838 31756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:56.261475 31756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:56.261700 31756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:56.262347 31756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:56.263252 31756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:56.263301 31756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.263343 31756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:56.263403 31756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.273041 31756 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:41421
I20260811 22:25:56.273051 31896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:41421 every 8 connection(s)
I20260811 22:25:56.274050 31756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:56.282866 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 31756
I20260811 22:25:56.282974 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:25:56.283290 24863 external_mini_cluster.cc:1428] Setting key 61fcc219c0c9a121829d5558d654e19a
Aug 11 22:25:56 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:25:56.290318 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:56.276106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:49309 (local address 127.24.71.254:35027)
0811 22:25:56.276270 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:56.276276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:56.277284 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 22:25:56.278798 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:56.278801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:56.278851 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:56.278949 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:56.280298 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.280879 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.282142 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.282455 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.282582 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:56.285774 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:56.285780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:56.285785 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:56.285819 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:56.287894 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:56.288470 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:56.288476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:56.288480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:56.288585 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:56.289028 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:56.289031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:56.289033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:56.289247 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:56.289364 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:56.289983 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 22:25:56.290134 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:25:56.291239 31899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:56.276580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.193:49309)
0811 22:25:56.277114 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:56.277163 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:56.278558 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:56.279130 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:56.279138 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:56.279591 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:56.280129 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:56.280143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.281025 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.281028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:56.281961 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:56.281972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.282629 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.282632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:56.282779 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:56.283552 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:56.283569 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:56.285527 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:56.288028 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:56.288034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:56.288049 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:56.288332 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:56.288726 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:56.288729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:56.288730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:56.288892 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:56.289391 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:56.289406 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:56.289688 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 22:25:56.289891 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:25:56.290271 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":138,"threads_started":1}
I20260811 22:25:56.292730 31897 heartbeater.cc:347] Connected to a master server at 127.24.71.254:35027
I20260811 22:25:56.293053 31897 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:56.293780 31897 heartbeater.cc:511] Master 127.24.71.254:35027 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:25:56.295784 31685 ts_manager.cc:194] Registered new tserver with Master: 35f107625eac4a55b7eb0301a96ff278 (127.24.71.193:41421)
I20260811 22:25:56.297078 31685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:49309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab.
Entry for principal oryx/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab.
I20260811 22:25:56.321144 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:35027
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab
--principal=oryx/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:56.442701 31903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:56.443009 31903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:56.443055 31903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:56.447294 31903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:56.447377 31903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:56.447441 31903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:56.452972 31903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab
--principal=oryx/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:35027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:56.454109 31903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:56.455196 31903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:56.465606 31911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:56.465606 31909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:56.465606 31908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:56.465909 31903 server_base.cc:1034] running on GCE node
I20260811 22:25:56.466503 31903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:56.467186 31903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:56.468349 31903 hybrid_clock.cc:648] HybridClock initialized: now 1786487156468330 us; error 31 us; skew 500 ppm
Aug 11 22:25:56 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:56.471359 31903 init.cc:377] Logged in from keytab as oryx/127.24.71.194@KRBTEST.COM (short username oryx)
I20260811 22:25:56.472399 31903 webserver.cc:460] Webserver started at http://127.24.71.194:33603/ using document root <none> and password file <none>
I20260811 22:25:56.473006 31903 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:56.473074 31903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:56.473269 31903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:56.475455 31903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "dbc9741b90d5474fa3268cca3bdbcfc3"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "85561d2fb2e8a17243b3161f7390ddbb"
server_key_iv: "6640bff87d17adac5e685df3c327a4bd"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.476105 31903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "dbc9741b90d5474fa3268cca3bdbcfc3"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "85561d2fb2e8a17243b3161f7390ddbb"
server_key_iv: "6640bff87d17adac5e685df3c327a4bd"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.480320 31903 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:25:56.483007 31920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.484467 31903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:56.484663 31903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "dbc9741b90d5474fa3268cca3bdbcfc3"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "85561d2fb2e8a17243b3161f7390ddbb"
server_key_iv: "6640bff87d17adac5e685df3c327a4bd"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.484797 31903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:56.509322 31903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:56.513227 31903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:56.513494 31903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:56.514218 31903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:56.515220 31903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:56.515300 31903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.515409 31903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:56.515461 31903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.526163 31903 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:46249
I20260811 22:25:56.526254 32045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:46249 every 8 connection(s)
I20260811 22:25:56.527284 31903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:56.530123 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 31903
I20260811 22:25:56.530213 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:25:56.530589 24863 external_mini_cluster.cc:1428] Setting key af7c370598c28b5869993c3559baf791
Aug 11 22:25:56 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:25:56.544246 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:56.529560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:35249 (local address 127.24.71.254:35027)
0811 22:25:56.529736 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:56.529739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:56.530894 (+ 1155us) server_negotiation.cc:407] Connection header received
0811 22:25:56.532403 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:56.532409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:56.532478 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:56.532559 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:56.534242 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.534875 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.536403 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.536712 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.536873 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:56.539740 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:56.539746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:56.539749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:56.539781 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:56.541620 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:56.542258 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:56.542262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:56.542263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:56.542353 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:56.542797 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:56.542803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:56.542805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:56.543048 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:56.543175 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:56.543901 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 22:25:56.544063 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.195@KRBTEST.COM" created.
I20260811 22:25:56.545349 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:56.529954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.194:35249)
0811 22:25:56.530703 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:56.530768 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:56.532197 (+ 1429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:56.532772 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:56.532781 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:56.533332 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:56.534064 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:56.534089 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.535036 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.535040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:56.536257 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:56.536268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.536964 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.536971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:56.537156 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:56.537849 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:56.537867 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:56.539532 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:56.541772 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:56.541777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:56.541793 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:56.542124 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:56.542520 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:56.542523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:56.542524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:56.542681 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:56.543213 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:56.543222 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:56.543603 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 22:25:56.543867 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 22:25:56.544252 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":179,"threads_started":1}
I20260811 22:25:56.546876 32046 heartbeater.cc:347] Connected to a master server at 127.24.71.254:35027
I20260811 22:25:56.547176 32046 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:56.547737 32046 heartbeater.cc:511] Master 127.24.71.254:35027 requested a full tablet report, sending...
I20260811 22:25:56.549050 31685 ts_manager.cc:194] Registered new tserver with Master: dbc9741b90d5474fa3268cca3bdbcfc3 (127.24.71.194:46249)
I20260811 22:25:56.549732 31685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:35249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab.
Entry for principal oryx/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab.
I20260811 22:25:56.566888 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:35027
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab
--principal=oryx/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:56.686000 32052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:56.686306 32052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:56.686354 32052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:56.690815 32052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:56.690893 32052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:56.690960 32052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:56.697067 32052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab
--principal=oryx/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:35027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:56.698328 32052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:56.699481 32052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:56.710987 32052 server_base.cc:1034] running on GCE node
W20260811 22:25:56.710922 32060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:56.710922 32058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:56.710922 32057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:56.711951 32052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:56.712766 32052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:56.713994 32052 hybrid_clock.cc:648] HybridClock initialized: now 1786487156713954 us; error 55 us; skew 500 ppm
Aug 11 22:25:56 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:56.717308 32052 init.cc:377] Logged in from keytab as oryx/127.24.71.195@KRBTEST.COM (short username oryx)
I20260811 22:25:56.718389 32052 webserver.cc:460] Webserver started at http://127.24.71.195:36759/ using document root <none> and password file <none>
I20260811 22:25:56.718986 32052 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:56.719053 32052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:56.719235 32052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:56.721580 32052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "fceaadf920814f1ab7560a10d13a8385"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "32e9c095e3fb2da4590f7bc8a441e726"
server_key_iv: "b6359c16137128aa8f5414ea7913e24f"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.722137 32052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "fceaadf920814f1ab7560a10d13a8385"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "32e9c095e3fb2da4590f7bc8a441e726"
server_key_iv: "b6359c16137128aa8f5414ea7913e24f"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.726246 32052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:25:56.729100 32070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.730719 32052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:56.730839 32052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "fceaadf920814f1ab7560a10d13a8385"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "32e9c095e3fb2da4590f7bc8a441e726"
server_key_iv: "b6359c16137128aa8f5414ea7913e24f"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.730940 32052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:56.756721 32052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:56.760380 32052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:56.760565 32052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:56.761149 32052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:56.762054 32052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:56.762104 32052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.762144 32052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:56.762161 32052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:56.772045 32052 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:34493
I20260811 22:25:56.772121 32196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:34493 every 8 connection(s)
I20260811 22:25:56.773041 32052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:56.775429 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 32052
I20260811 22:25:56.775589 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:25:56.776002 24863 external_mini_cluster.cc:1428] Setting key 18c3eabfc9d1078e732551e28e6bcd0c
Aug 11 22:25:56 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:25:56.788879 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:56.775119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:60763 (local address 127.24.71.254:35027)
0811 22:25:56.775271 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:56.775276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:56.776389 (+ 1113us) server_negotiation.cc:407] Connection header received
0811 22:25:56.778095 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:56.778098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:56.778155 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:56.778235 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:56.779590 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.780227 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.781424 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.781654 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.781770 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:56.784730 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:56.784734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:56.784736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:56.784766 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:56.786252 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:56.786969 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:56.786975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:56.786977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:56.787081 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:56.787478 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:56.787481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:56.787482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:56.787701 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:56.787791 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:56.788560 (+ 769us) server_negotiation.cc:299] Negotiation successful
0811 22:25:56.788690 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:25:56.790019 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:56.775584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.195:60763)
0811 22:25:56.776174 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:56.776248 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:56.777879 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:56.778441 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:56.778448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:56.778927 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:56.779443 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:56.779457 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.780371 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.780374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:56.781281 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:56.781291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:56.781817 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:56.781822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:56.782003 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:56.782652 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:56.782670 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:56.784420 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:56.786396 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:56.786403 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:56.786424 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:56.786749 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:56.787188 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:56.787191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:56.787192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:56.787365 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:56.787809 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:56.787815 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:56.788233 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 22:25:56.788516 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 22:25:56.788843 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":142,"threads_started":1}
I20260811 22:25:56.791354 32197 heartbeater.cc:347] Connected to a master server at 127.24.71.254:35027
I20260811 22:25:56.791656 32197 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:56.792428 32197 heartbeater.cc:511] Master 127.24.71.254:35027 requested a full tablet report, sending...
I20260811 22:25:56.793733 31685 ts_manager.cc:194] Registered new tserver with Master: fceaadf920814f1ab7560a10d13a8385 (127.24.71.195:34493)
I20260811 22:25:56.794497 31685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:60763
I20260811 22:25:56.803361 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.71.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.253.keytab.
Entry for principal oryx/127.24.71.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.253.keytab.
I20260811 22:25:56.836858 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.253:35393
--webserver_interface=127.24.71.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.253.keytab
--principal=oryx/127.24.71.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:35027,127.24.71.253:35393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:25:56.961633 32202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:56.961990 32202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:56.962040 32202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:56.966902 32202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:56.966985 32202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:56.967005 32202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:56.967023 32202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:56.967082 32202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:56.972800 32202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:35027,127.24.71.253:35393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.253.keytab
--principal=oryx/127.24.71.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.253:35393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.71.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
W20260811 22:25:56.973069 32202 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.71.254:35027,127.24.71.253:35393'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 22:25:56.974032 32202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:56.975147 32202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:56.984318 32207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:56.984304 32208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:56.984428 32202 server_base.cc:1034] running on GCE node
W20260811 22:25:56.984280 32210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:56.985147 32202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:56.986378 32202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:56.987598 32202 hybrid_clock.cc:648] HybridClock initialized: now 1786487156987578 us; error 38 us; skew 500 ppm
Aug 11 22:25:56 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:56.990797 32202 init.cc:377] Logged in from keytab as oryx/127.24.71.253@KRBTEST.COM (short username oryx)
I20260811 22:25:56.991765 32202 webserver.cc:460] Webserver started at http://127.24.71.253:37553/ using document root <none> and password file <none>
I20260811 22:25:56.992406 32202 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:56.992483 32202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:56.992663 32202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:56.995057 32202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data/instance:
uuid: "8e796f4ce0654b669a4ed5f8ace642f8"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "a9aac63b1b75c0377ebb53d1e065eec4"
server_key_iv: "a9a6a9f2c3509e64652d1b1b5f0074d7"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.995625 32202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal/instance:
uuid: "8e796f4ce0654b669a4ed5f8ace642f8"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "a9aac63b1b75c0377ebb53d1e065eec4"
server_key_iv: "a9a6a9f2c3509e64652d1b1b5f0074d7"
server_key_version: "encryptionkey@0"
I20260811 22:25:56.999886 32202 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:25:57.002624 32217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.004288 32202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:57.004470 32202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal
uuid: "8e796f4ce0654b669a4ed5f8ace642f8"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "a9aac63b1b75c0377ebb53d1e065eec4"
server_key_iv: "a9a6a9f2c3509e64652d1b1b5f0074d7"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.004568 32202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:57.033174 32202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:57.036952 32202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:57.037124 32202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:57.044682 32202 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.253:35393
I20260811 22:25:57.044699 32276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.253:35393 every 8 connection(s)
I20260811 22:25:57.045784 32202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
I20260811 22:25:57.048967 32277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:57.053040 32277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:25:57.055823 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 32202
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.253@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.71.253@KRBTEST.COM
I20260811 22:25:57.069226 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.055142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.24.71.254:35027)
0811 22:25:57.055305 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.055309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.056480 (+ 1171us) server_negotiation.cc:407] Connection header received
0811 22:25:57.057651 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.057655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.057725 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.057838 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.058919 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.059462 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.060607 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.060850 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.060972 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.064257 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.064261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.064263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.064362 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.066231 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.067019 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.067026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.067031 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.067137 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.067674 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.067678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.067679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.068071 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.068182 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.068872 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.069015 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:25:57.070075 32279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.055594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:54808)
0811 22:25:57.056269 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.056335 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.057417 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.057933 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.057946 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.058350 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.058752 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.058768 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.059617 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.059620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.060468 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.060478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.061023 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.061027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.061208 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.061893 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.061918 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.064040 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.066518 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.066527 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.066546 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.066884 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.067254 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.067258 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.067260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.067535 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.068188 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.068194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.068572 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.068768 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.069135 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":3,"spinlock_wait_cycles":11776,"thread_start_us":214,"threads_started":1}
I20260811 22:25:57.070081 32281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.057085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.24.71.253:35393)
0811 22:25:57.057496 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.057509 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.057558 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:25:57.057719 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.057723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.057917 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.058118 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.059002 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.059957 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.060821 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.061226 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.061348 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.064652 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.064663 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.064687 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.064709 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.066920 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.067525 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.067536 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.067538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.067668 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.068024 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.068028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.068029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.068496 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.068648 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.068942 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.069376 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":280,"server-negotiator.queue_time_us":290,"spinlock_wait_cycles":2432,"thread_start_us":147,"threads_started":1}
I20260811 22:25:57.072441 32277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487156, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.253@KRBTEST.COM for oryx/127.24.71.253@KRBTEST.COM
I20260811 22:25:57.081517 32279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.072990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.0.0.1:51624)
0811 22:25:57.073102 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.073117 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.073197 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.073553 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.073555 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.073691 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.073782 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.073787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.074507 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.074509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.075019 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.075026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.076268 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.076271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.076404 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.076939 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.076951 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.078172 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.079894 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.079899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.079903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.080106 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.080389 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.080391 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.080393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.080506 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.080924 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.080928 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.081031 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.081167 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.081305 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 22:25:57.081517 32281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.073173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51624 (local address 127.24.71.253:35393)
0811 22:25:57.073306 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.073311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.073325 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:57.073364 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.073366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.073400 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.073468 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.073889 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.074411 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.075145 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.076143 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.076245 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.078327 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.078331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.078333 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.078359 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.079721 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.080210 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.080213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.080214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.080290 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.080624 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.080628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.080629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.080803 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.080923 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.081222 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.081362 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":61}
I20260811 22:25:57.084573 32277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: Bootstrap starting.
I20260811 22:25:57.087019 32277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:57.087973 32277 log.cc:824] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:57.090433 32277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: No bootstrap required, opened a new log
I20260811 22:25:57.094087 32277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } }
I20260811 22:25:57.094347 32277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:57.094440 32277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e796f4ce0654b669a4ed5f8ace642f8, State: Initialized, Role: FOLLOWER
I20260811 22:25:57.094959 32277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } }
I20260811 22:25:57.095503 32283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } } }
I20260811 22:25:57.095619 32283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:25:57.096416 32277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:25:57.102452 32294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:25:57.102568 32294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:25:57.105204 32202 master_runner.cc:428] Detected that this master 8e796f4ce0654b669a4ed5f8ace642f8 is joining an existing cluster
I20260811 22:25:57.105291 32202 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.71.253:35393
I20260811 22:25:57.117885 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.107945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.24.71.254:35027)
0811 22:25:57.108108 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.108113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.108307 (+ 194us) server_negotiation.cc:407] Connection header received
0811 22:25:57.108495 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.108498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.108551 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.108610 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.109355 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.110151 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.111321 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.111697 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.111815 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.113561 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.113565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.113568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.113608 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.115473 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.116206 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.116210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.116212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.116323 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.116746 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.116750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.116752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.117020 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.117123 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.117556 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.117708 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:25:57.118171 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.108501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.0.0.1:51626)
0811 22:25:57.108847 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.108859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.108986 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.109346 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.109349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.109594 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.109721 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.109730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.110746 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.110750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.111552 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.111560 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.112282 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.112286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.112407 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.113012 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.113027 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.113579 (+ 552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.116277 (+ 2698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.116293 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.116295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.116554 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.116866 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.116872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.116874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.116972 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.117447 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.117451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.117559 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.117700 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.117908 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":97,"threads_started":1}
I20260811 22:25:57.118649 32281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.108508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51626 (local address 127.24.71.253:35393)
0811 22:25:57.108648 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.108652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.108943 (+ 291us) server_negotiation.cc:407] Connection header received
0811 22:25:57.109090 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.109093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.109134 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.109187 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.109850 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.110619 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.111671 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.112131 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.112268 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.113716 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.113720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.113722 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.113744 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.116155 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.116662 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.116669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.116675 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.116757 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.117070 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.117076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.117078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.117329 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.117457 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.117751 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.118472 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":208,"server-negotiator.queue_time_us":59,"spinlock_wait_cycles":896}
I20260811 22:25:57.118810 32300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.107819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:54824)
0811 22:25:57.108197 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.108213 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.108338 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.108775 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.108778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.109024 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.109182 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.109190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.110274 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.110281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.111185 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.111196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.111820 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.111824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.112056 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.112793 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.112815 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.113389 (+ 574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.115612 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.115617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.115621 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.116078 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.116458 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.116461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.116463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.116562 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.117172 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.117175 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.117341 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.118162 (+ 821us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.118562 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":264,"spinlock_wait_cycles":768,"thread_start_us":151,"threads_started":1}
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487155, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.254@KRBTEST.COM for oryx/127.24.71.253@KRBTEST.COM
WARNING: no policy specified for oryx/127.24.71.252@KRBTEST.COM; defaulting to no policy
I20260811 22:25:57.136787 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.124760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.0.0.1:51638)
0811 22:25:57.125640 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.125699 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.125852 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.126334 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.126339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.126845 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.126993 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.127002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.127814 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.127818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.128755 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.128769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.129257 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.129261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.129373 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.130108 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.130128 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.132381 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.134559 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.134570 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.134578 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.134933 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.135399 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.135404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.135406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.135509 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.136030 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.136041 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.136233 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.136389 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.136634 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":106,"threads_started":1}
I20260811 22:25:57.136817 32281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.124804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51638 (local address 127.24.71.253:35393)
0811 22:25:57.124947 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.124955 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.125795 (+ 840us) server_negotiation.cc:407] Connection header received
0811 22:25:57.126029 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.126032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.126089 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.126140 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.127137 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.127683 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.128889 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.129128 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.129213 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.132553 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.132557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.132561 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.132591 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.134372 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.135138 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.135143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.135145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.135236 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.135605 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.135609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.135611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.135872 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.135981 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.136458 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.136643 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.252@KRBTEST.COM" created.
I20260811 22:25:57.138582 31685 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.71.253:35393: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8"
member_type: NON_VOTER
last_known_addr {
host: "127.24.71.253"
port: 35393
}
attrs {
promote: true
}
}
dest_uuid: "9918892880914ddf8b203bc656c08730"
cas_config_opid_index: -1
I20260811 22:25:57.139358 31685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } }
I20260811 22:25:57.142374 32247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:57.142341 32306 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8e796f4ce0654b669a4ed5f8ace642f8 (127.24.71.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } } }
I20260811 22:25:57.143972 32247 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Refusing update from remote peer 9918892880914ddf8b203bc656c08730: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 22:25:57.144644 32306 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.71.253:35393
I20260811 22:25:57.144917 32306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } } }
I20260811 22:25:57.145047 32306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:57.145233 32202 master.cc:537] Master@127.24.71.253:35393 shutting down...
I20260811 22:25:57.145237 32306 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:25:57.149338 31672 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 22:25:57.150411 32202 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:57.150739 32202 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 22:25:57.150830 31672 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 -> Peer 8e796f4ce0654b669a4ed5f8ace642f8 (127.24.71.253:35393): Couldn't send request to peer 8e796f4ce0654b669a4ed5f8ace642f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:25:57.150872 32202 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.252.keytab.
Entry for principal oryx/127.24.71.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.252.keytab.
I20260811 22:25:57.163504 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.252:34933
--webserver_interface=127.24.71.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.252.keytab
--principal=oryx/127.24.71.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:35027,127.24.71.253:35393,127.24.71.252:34933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:25:57.166862 32202 master.cc:552] Master@127.24.71.253:35393 shutdown complete.
I20260811 22:25:57.166935 32202 master_runner.cc:305] Clearing existing system tablet
I20260811 22:25:57.167207 32202 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:57.167270 32202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:57.168823 32202 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 22:25:57.169888 32314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.170188 32202 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:57.170318 32202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:57.170353 32202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal
uuid: "8e796f4ce0654b669a4ed5f8ace642f8"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "a9aac63b1b75c0377ebb53d1e065eec4"
server_key_iv: "a9a6a9f2c3509e64652d1b1b5f0074d7"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.172694 32202 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:57.175232 32202 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:25:57.175333 32202 log.cc:1196] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 22:25:57.175604 32202 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: Deleting consensus metadata
I20260811 22:25:57.175793 32202 master_runner.cc:315] Copying system tablet from 127.24.71.254:35027
I20260811 22:25:57.177235 32202 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.254:35027
I20260811 22:25:57.187266 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.177970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:54838)
0811 22:25:57.178312 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.178328 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.178467 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.178896 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.178902 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.179126 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.179267 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.179275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.180448 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.180452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.181378 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.181389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.182093 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.182097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.182202 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.182882 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.182898 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.183340 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.185509 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.185512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.185515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.185790 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.186113 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.186116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.186118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.186216 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.186680 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.186683 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.186791 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.186904 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.187059 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":91,"threads_started":1}
I20260811 22:25:57.187294 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.178077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.24.71.254:35027)
0811 22:25:57.178217 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.178222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.178429 (+ 207us) server_negotiation.cc:407] Connection header received
0811 22:25:57.178693 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.178696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.178754 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.178862 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.179407 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.180295 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.181530 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.181945 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.182085 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.183519 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.183524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.183527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.183558 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.185370 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.185900 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.185906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.185909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.186007 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.186314 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.186318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.186319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.186570 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.186674 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.186980 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.187111 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:25:57.188772 31710 tablet_copy_service.cc:140] P 9918892880914ddf8b203bc656c08730: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8e796f4ce0654b669a4ed5f8ace642f8 ({username='oryx', principal='oryx/127.24.71.253@KRBTEST.COM'} at 127.0.0.1:54838)
I20260811 22:25:57.188979 31710 tablet_copy_service.cc:161] P 9918892880914ddf8b203bc656c08730: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8e796f4ce0654b669a4ed5f8ace642f8 at {username='oryx', principal='oryx/127.24.71.253@KRBTEST.COM'} at 127.0.0.1:54838: session id = 8e796f4ce0654b669a4ed5f8ace642f8-00000000000000000000000000000000
I20260811 22:25:57.191020 31710 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:25:57.193066 32202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:25:57.196080 32202 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: tablet copy: Starting download of 0 data blocks...
I20260811 22:25:57.196426 32202 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: tablet copy: Starting download of 1 WAL segments...
I20260811 22:25:57.198621 32202 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:25:57.200640 31710 tablet_copy_service.cc:342] P 9918892880914ddf8b203bc656c08730: Request end of tablet copy session 8e796f4ce0654b669a4ed5f8ace642f8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.71.253@KRBTEST.COM'} at 127.0.0.1:54838
I20260811 22:25:57.200783 31710 tablet_copy_service.cc:433] P 9918892880914ddf8b203bc656c08730: ending tablet copy session 8e796f4ce0654b669a4ed5f8ace642f8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8e796f4ce0654b669a4ed5f8ace642f8
I20260811 22:25:57.208824 32202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:57.210764 32324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:57.210912 32321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:57.210999 32202 server_base.cc:1034] running on GCE node
W20260811 22:25:57.211136 32322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:57.211337 32202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:57.211681 32202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:57.212836 32202 hybrid_clock.cc:648] HybridClock initialized: now 1786487157212784 us; error 69 us; skew 500 ppm
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:57.214704 32202 init.cc:377] Logged in from keytab as oryx/127.24.71.253@KRBTEST.COM (short username oryx)
I20260811 22:25:57.215217 32202 webserver.cc:460] Webserver started at http://127.24.71.253:42935/ using document root <none> and password file <none>
I20260811 22:25:57.215497 32202 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:57.215550 32202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:57.217057 32202 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:57.218170 32331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.218492 32202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:57.218570 32202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal
uuid: "8e796f4ce0654b669a4ed5f8ace642f8"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "a9aac63b1b75c0377ebb53d1e065eec4"
server_key_iv: "a9a6a9f2c3509e64652d1b1b5f0074d7"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.218655 32202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:57.244433 32202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:57.246840 32202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:57.252286 32202 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.253:35393
I20260811 22:25:57.252319 32390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.253:35393 every 8 connection(s)
I20260811 22:25:57.253103 32202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
I20260811 22:25:57.255199 32391 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:25:57.256338 32391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: Bootstrap starting.
I20260811 22:25:57.268476 32391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:25:57.268858 32391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: Bootstrap complete.
I20260811 22:25:57.269567 32391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } }
I20260811 22:25:57.269713 32391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8e796f4ce0654b669a4ed5f8ace642f8, State: Initialized, Role: LEARNER
I20260811 22:25:57.269824 32391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } }
I20260811 22:25:57.270220 32394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } } }
I20260811 22:25:57.270323 32391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:57.270452 32394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:25:57.273805 32406 catalog_manager.cc:1259] Loaded cluster ID: c8e20ee46ee141e7bac9731a76061819
I20260811 22:25:57.273936 32406 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: loading cluster ID for follower catalog manager: success
I20260811 22:25:57.276264 32406 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: acquiring CA information for follower catalog manager: success
I20260811 22:25:57.277757 32406 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:25:57.300143 31897 heartbeater.cc:503] Master 127.24.71.254:35027 was elected leader, sending a full tablet report...
W20260811 22:25:57.313249 32311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:57.313614 32311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:57.313706 32311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:57.318836 32311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:25:57.318953 32311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:57.318997 32311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:57.319087 32311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:25:57.319139 32311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:25:57.325766 32311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:35027,127.24.71.253:35393,127.24.71.252:34933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.252.keytab
--principal=oryx/127.24.71.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.252:34933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.71.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:57.327085 32311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:57.328365 32311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:57.338567 32412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:57.338595 32415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:57.338594 32411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:57.338692 32311 server_base.cc:1034] running on GCE node
I20260811 22:25:57.339430 32311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:57.340631 32311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:57.341800 32311 hybrid_clock.cc:648] HybridClock initialized: now 1786487157341760 us; error 53 us; skew 500 ppm
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:57.344734 32311 init.cc:377] Logged in from keytab as oryx/127.24.71.252@KRBTEST.COM (short username oryx)
I20260811 22:25:57.345695 32311 webserver.cc:460] Webserver started at http://127.24.71.252:38591/ using document root <none> and password file <none>
I20260811 22:25:57.346275 32311 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:57.346367 32311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:57.346618 32311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:25:57.348810 32311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data/instance:
uuid: "42a677fee666457192ec1f4407ed69d9"
format_stamp: "Formatted at 2026-08-11 22:25:57 on dist-test-slave-mvvj"
server_key: "2b955b9d9ca072582d60d8949659d91e"
server_key_iv: "ef9ffe703cb4484a3e2bd448d73bc734"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.349401 32311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal/instance:
uuid: "42a677fee666457192ec1f4407ed69d9"
format_stamp: "Formatted at 2026-08-11 22:25:57 on dist-test-slave-mvvj"
server_key: "2b955b9d9ca072582d60d8949659d91e"
server_key_iv: "ef9ffe703cb4484a3e2bd448d73bc734"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.353720 32311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 22:25:57.356679 32423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.358319 32311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:25:57.358516 32311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal
uuid: "42a677fee666457192ec1f4407ed69d9"
format_stamp: "Formatted at 2026-08-11 22:25:57 on dist-test-slave-mvvj"
server_key: "2b955b9d9ca072582d60d8949659d91e"
server_key_iv: "ef9ffe703cb4484a3e2bd448d73bc734"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.358664 32311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:57.369944 32311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:57.374349 32311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:57.374614 32311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:57.383088 32311 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.252:34933
I20260811 22:25:57.383528 32478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.252:34933 every 8 connection(s)
I20260811 22:25:57.384263 32311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
I20260811 22:25:57.387409 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:57.392376 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 32311
I20260811 22:25:57.392112 32479 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.252@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.71.252@KRBTEST.COM
I20260811 22:25:57.408747 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.395232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.24.71.254:35027)
0811 22:25:57.395400 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.395405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.395543 (+ 138us) server_negotiation.cc:407] Connection header received
0811 22:25:57.396129 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.396133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.396186 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.396298 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.397591 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.398560 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.400170 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.400556 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.400686 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.404046 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.404053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.404059 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.404103 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.405844 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.406667 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.406671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.406672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.406761 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.407307 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.407311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.407313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.407551 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.407681 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.408374 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.408537 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:25:57.409724 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.395022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:54844)
0811 22:25:57.395381 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.395434 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.395889 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.396464 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.396470 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.396977 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.397381 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.397406 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.398744 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.398748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.400027 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.400040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.400758 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.400762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.400914 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.401674 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.401704 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.403777 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.405998 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.406007 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.406017 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.406486 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.406874 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.406878 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.406879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.407086 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.407691 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.407697 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.408075 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.408292 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.408704 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":92,"threads_started":1}
I20260811 22:25:57.409725 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.394301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40812 (local address 127.24.71.252:34933)
0811 22:25:57.394765 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.394781 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.394817 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:25:57.395879 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.395912 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.396171 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.396364 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.397595 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.399019 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.400315 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.400965 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.401058 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.404490 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.404496 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.404509 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.404531 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.406453 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.407026 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.407031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.407032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.407121 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.407615 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.407620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.407622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.408063 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.408226 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.408535 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.409214 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":255,"server-negotiator.queue_time_us":318,"thread_start_us":143,"threads_started":1}
I20260811 22:25:57.411338 32479 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.252@KRBTEST.COM for oryx/127.24.71.253@KRBTEST.COM
I20260811 22:25:57.421753 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.411952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.0.0.1:51640)
0811 22:25:57.412148 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.412167 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.412258 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.412624 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.412626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.412779 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.412885 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.412890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.413804 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.413807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.414410 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.414421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.414955 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.414958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.415044 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.415696 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.415710 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.417284 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.419732 (+ 2448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.419734 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.419736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.420062 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.420517 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.420520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.420522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.420620 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.421170 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.421173 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.421282 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.421459 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.421599 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 22:25:57.421868 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.412118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.24.71.253:35393)
0811 22:25:57.412410 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.412413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.412427 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:57.412466 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.412468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.412510 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.412585 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.412996 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.413681 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.414562 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.414813 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.414916 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.417551 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.417555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.417557 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.417584 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.419607 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.420221 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.420226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.420228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.420391 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.420769 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.420774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.420776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.420993 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.421084 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.421584 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.421728 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":108,"threads_started":1}
I20260811 22:25:57.422734 32479 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.252@KRBTEST.COM for oryx/127.24.71.252@KRBTEST.COM
I20260811 22:25:57.431452 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.423287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:34933 (local address 127.0.0.1:40828)
0811 22:25:57.423500 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.423517 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.423642 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.423925 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.423928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.424104 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.424185 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.424191 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.425045 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.425048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.425551 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.425559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.426024 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.426027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.426096 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.426680 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.426692 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.428085 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.429845 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.429848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.429850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.430054 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.430424 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.430429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.430431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.430515 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.430904 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.430907 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.431001 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.431113 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.431261 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 22:25:57.431452 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.423342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40828 (local address 127.24.71.252:34933)
0811 22:25:57.423500 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.423506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.423590 (+ 84us) server_negotiation.cc:407] Connection header received
0811 22:25:57.423733 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.423736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.423780 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.423902 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.424297 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.424895 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.425672 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.425909 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.426023 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.428239 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.428242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.428244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.428273 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.429704 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.430161 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.430165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.430168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.430242 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.430613 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.430616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.430618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.430803 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.430916 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.431157 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.431334 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":68}
I20260811 22:25:57.435199 32479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Bootstrap starting.
I20260811 22:25:57.437798 32479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:57.438551 32479 log.cc:824] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:57.440716 32479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: No bootstrap required, opened a new log
I20260811 22:25:57.443779 32479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } }
I20260811 22:25:57.444070 32479 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:57.444110 32479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a677fee666457192ec1f4407ed69d9, State: Initialized, Role: FOLLOWER
I20260811 22:25:57.444504 32479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } }
I20260811 22:25:57.444972 32487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } } }
I20260811 22:25:57.445106 32487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:25:57.445797 32479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:25:57.450898 32499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:25:57.451046 32499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:25:57.454157 32311 master_runner.cc:428] Detected that this master 42a677fee666457192ec1f4407ed69d9 is joining an existing cluster
I20260811 22:25:57.454243 32311 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.71.252:34933
I20260811 22:25:57.467083 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 31756
I20260811 22:25:57.478964 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:41421
--local_ip_for_outbound_sockets=127.24.71.193
--tserver_master_addrs=127.24.71.254:35027
--webserver_port=45731
--webserver_interface=127.24.71.193
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab
--principal=oryx/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.71.254:35027,127.24.71.253:35393
--tserver_master_addrs=127.24.71.254:35027,127.24.71.253:35393,127.24.71.252:34933 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:25:57.494411 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.457052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:34933 (local address 127.0.0.1:40830)
0811 22:25:57.458172 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.458188 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.458361 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.461696 (+ 3335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.461699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.462038 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.462184 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.462192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.463376 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.463379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.464299 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.464314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.465019 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.465024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.465117 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.465698 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.465715 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.466270 (+ 555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.491970 (+ 25700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.491975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.491979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.492268 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.492584 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.492587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.492589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.492661 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.493124 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.493127 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.493876 (+ 749us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.493998 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.494182 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"thread_start_us":122,"threads_started":1}
I20260811 22:25:57.494975 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.460632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.24.71.252:34933)
0811 22:25:57.460950 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.460956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.460972 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:57.461016 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.461018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.461065 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.461149 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.462295 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.463137 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.464427 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.464767 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.464843 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.466399 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.466404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.466407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.466427 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.491772 (+ 25345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.492412 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.492416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.492418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.492497 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.492742 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.492745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.492747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.492991 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.493143 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.494690 (+ 1547us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.494823 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":239}
I20260811 22:25:57.495041 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.460813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54858 (local address 127.24.71.254:35027)
0811 22:25:57.461349 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.461354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.461368 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:57.461412 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.461415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.461472 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.461553 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.463549 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.464383 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.465661 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.466003 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.466081 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.489397 (+ 23316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.489403 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.489406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.489452 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.492806 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.493560 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.493564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.493566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.493667 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.494004 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.494008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.494010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.494240 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.494313 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.494720 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.494866 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437}
I20260811 22:25:57.495558 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.457077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:54858)
0811 22:25:57.460827 (+ 3750us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.460844 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.460954 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.461684 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.461687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.462517 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.463392 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.463404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.464535 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.464538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.465511 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.465522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.466236 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.466239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.466352 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.467592 (+ 1240us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.467608 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.488846 (+ 21238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.492984 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.492988 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.492990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.493428 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.493781 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.493784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.493787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.493873 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.494405 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.494408 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.494501 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.495210 (+ 709us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.495368 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3670,"mutex_wait_us":1,"spinlock_wait_cycles":8576,"thread_start_us":139,"threads_started":1}
I20260811 22:25:57.498178 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.457793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.24.71.253:35393)
0811 22:25:57.457958 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.457964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.461230 (+ 3266us) server_negotiation.cc:407] Connection header received
0811 22:25:57.461431 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.461436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.461495 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.461549 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.466605 (+ 5056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.467417 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.468775 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.469144 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.469218 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.489952 (+ 20734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.489973 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.489979 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.490026 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.491850 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.496828 (+ 4978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.496833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.496835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.496937 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.497229 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.497232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.497234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.497463 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.497548 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.497875 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.498022 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:25:57.498739 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.457626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.0.0.1:51652)
0811 22:25:57.461145 (+ 3519us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.461157 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.461309 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.461886 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.461889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.462125 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.462260 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.462268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.467703 (+ 5435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.467706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.468628 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.468639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.469404 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.469408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.469513 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.473099 (+ 3586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.473126 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.489720 (+ 16594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.496433 (+ 6713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.496436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.496438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.496683 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.497049 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.497052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.497054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.497134 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.497608 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.497610 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.497693 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.498407 (+ 714us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.498564 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3455,"thread_start_us":82,"threads_started":1}
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487155, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.254@KRBTEST.COM for oryx/127.24.71.252@KRBTEST.COM
I20260811 22:25:57.510201 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.500690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:34933 (local address 127.0.0.1:40836)
0811 22:25:57.500809 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.500826 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.500915 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.501231 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.501235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.501446 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.501577 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.501584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.502627 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.502631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.503515 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.503526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.504172 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.504176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.504282 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.504807 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.504823 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.506482 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.508513 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.508517 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.508519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.508782 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.509095 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.509099 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.509101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.509198 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.509620 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.509624 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.509743 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.509856 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.510011 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 22:25:57.510201 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.500783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40836 (local address 127.24.71.252:34933)
0811 22:25:57.500904 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.500908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.500922 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:57.501035 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.501038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.501093 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.501168 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.501701 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.502500 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.503655 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.504047 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.504151 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.506639 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.506643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.506646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.506672 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.508393 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.508890 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.508893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.508895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.508993 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.509295 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.509299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.509300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.509515 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.509616 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.509923 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.510053 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:25:57.511915 31685 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.71.252:34933: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "42a677fee666457192ec1f4407ed69d9"
member_type: NON_VOTER
last_known_addr {
host: "127.24.71.252"
port: 34933
}
attrs {
promote: true
}
}
dest_uuid: "9918892880914ddf8b203bc656c08730"
cas_config_opid_index: 5
I20260811 22:25:57.512588 31685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } }
I20260811 22:25:57.513751 32308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 42a677fee666457192ec1f4407ed69d9 (127.24.71.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.515269 32308 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.71.252:34933
I20260811 22:25:57.515452 32308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.515566 32308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:57.516625 32311 master.cc:537] Master@127.24.71.252:34933 shutting down...
I20260811 22:25:57.516841 32450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:57.518343 32450 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Refusing update from remote peer 9918892880914ddf8b203bc656c08730: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:25:57.519369 32306 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:57.522115 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.515922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.0.0.1:51654)
0811 22:25:57.517279 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.517304 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.517395 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.517833 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.517836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.517854 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:57.517945 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.517950 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.519110 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.519113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.520447 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.520459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.521617 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.521624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.521745 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.521817 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.521896 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.521980 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1275}
W20260811 22:25:57.522151 31674 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 -> Peer 42a677fee666457192ec1f4407ed69d9 (127.24.71.252:34933): Couldn't send request to peer 42a677fee666457192ec1f4407ed69d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:25:57.522541 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.516039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51654 (local address 127.24.71.253:35393)
0811 22:25:57.516168 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.516173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.517487 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 22:25:57.517549 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.517552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.517616 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.517700 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:57.518071 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.518942 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.520598 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.521246 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.521343 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.521503 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.522334 (+ 831us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.522415 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:25:57.523360 32363 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 LEARNER]: Refusing update from remote peer 9918892880914ddf8b203bc656c08730: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:25:57.523916 32306 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 22:25:57.526412 32363 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 42a677fee666457192ec1f4407ed69d9 (127.24.71.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.529721 32394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9918892880914ddf8b203bc656c08730. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.529855 32394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:25:57.529985 32394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: NON_VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: true } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.530092 32394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:25:57.531924 32307 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: attempting to promote NON_VOTER 8e796f4ce0654b669a4ed5f8ace642f8 to VOTER
I20260811 22:25:57.532469 32307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } }
I20260811 22:25:57.534219 32363 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 LEARNER]: Refusing update from remote peer 9918892880914ddf8b203bc656c08730: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 22:25:57.534636 32306 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 22:25:57.534934 31674 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 -> Peer 42a677fee666457192ec1f4407ed69d9 (127.24.71.252:34933): Couldn't send request to peer 42a677fee666457192ec1f4407ed69d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:25:57.535193 32311 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:57.535485 32311 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:57.535586 32311 tablet_replica.cc:330] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: stopping tablet replica
I20260811 22:25:57.536665 32308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8e796f4ce0654b669a4ed5f8ace642f8 (127.24.71.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.540114 32363 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8e796f4ce0654b669a4ed5f8ace642f8 (127.24.71.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.541971 32308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.542079 32306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.542289 32306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:57.542104 32308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:57.541801 32511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.543377 32511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:25:57.552101 32046 heartbeater.cc:503] Master 127.24.71.254:35027 was elected leader, sending a full tablet report...
I20260811 22:25:57.555606 32311 master.cc:552] Master@127.24.71.252:34933 shutdown complete.
I20260811 22:25:57.555678 32311 master_runner.cc:305] Clearing existing system tablet
I20260811 22:25:57.555977 32311 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:57.556046 32311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:57.557489 32311 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:57.558511 32513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.558807 32311 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:57.559145 32311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:25:57.559199 32311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal
uuid: "42a677fee666457192ec1f4407ed69d9"
format_stamp: "Formatted at 2026-08-11 22:25:57 on dist-test-slave-mvvj"
server_key: "2b955b9d9ca072582d60d8949659d91e"
server_key_iv: "ef9ffe703cb4484a3e2bd448d73bc734"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.561545 32311 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:57.564227 32311 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:25:57.564337 32311 log.cc:1196] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 22:25:57.564603 32311 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Deleting consensus metadata
I20260811 22:25:57.564781 32311 master_runner.cc:315] Copying system tablet from 127.24.71.254:35027
I20260811 22:25:57.566062 32311 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: tablet copy: Beginning tablet copy session from remote peer at address 127.24.71.254:35027
I20260811 22:25:57.575924 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.566699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:54870)
0811 22:25:57.567045 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.567062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.567194 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.567533 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.567537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.567778 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.567932 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.567941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.569133 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.569137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.569985 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.569999 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.570673 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.570677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.570775 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.571415 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.571431 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.571989 (+ 558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.574082 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.574088 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.574091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.574382 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.574707 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.574710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.574712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.574818 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.575306 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.575310 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.575420 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.575538 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.575702 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":101,"threads_started":1}
I20260811 22:25:57.575934 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.566826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54870 (local address 127.24.71.254:35027)
0811 22:25:57.566974 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.566979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.567159 (+ 180us) server_negotiation.cc:407] Connection header received
0811 22:25:57.567308 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.567313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.567381 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.567496 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.568096 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.568892 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.570185 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.570528 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.570642 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.572151 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.572157 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.572162 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.572196 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.573945 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.574496 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.574507 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.574508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.574603 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.574931 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.574936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.574941 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.575196 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.575302 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.575614 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.575742 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:25:57.576931 31710 tablet_copy_service.cc:140] P 9918892880914ddf8b203bc656c08730: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 42a677fee666457192ec1f4407ed69d9 ({username='oryx', principal='oryx/127.24.71.252@KRBTEST.COM'} at 127.0.0.1:54870)
I20260811 22:25:57.577068 31710 tablet_copy_service.cc:161] P 9918892880914ddf8b203bc656c08730: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 42a677fee666457192ec1f4407ed69d9 at {username='oryx', principal='oryx/127.24.71.252@KRBTEST.COM'} at 127.0.0.1:54870: session id = 42a677fee666457192ec1f4407ed69d9-00000000000000000000000000000000
I20260811 22:25:57.578322 31710 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:25:57.579859 32311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:25:57.582984 32311 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: tablet copy: Starting download of 0 data blocks...
I20260811 22:25:57.583432 32311 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: tablet copy: Starting download of 1 WAL segments...
I20260811 22:25:57.585966 32311 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:25:57.588304 31710 tablet_copy_service.cc:342] P 9918892880914ddf8b203bc656c08730: Request end of tablet copy session 42a677fee666457192ec1f4407ed69d9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.71.252@KRBTEST.COM'} at 127.0.0.1:54870
I20260811 22:25:57.588411 31710 tablet_copy_service.cc:433] P 9918892880914ddf8b203bc656c08730: ending tablet copy session 42a677fee666457192ec1f4407ed69d9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 42a677fee666457192ec1f4407ed69d9
I20260811 22:25:57.598213 32311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:57.600483 32524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:57.600829 32311 server_base.cc:1034] running on GCE node
W20260811 22:25:57.600867 32521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:57.604030 32522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:57.604344 32311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:57.604701 32311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:57.605873 32311 hybrid_clock.cc:648] HybridClock initialized: now 1786487157605843 us; error 53 us; skew 500 ppm
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:57.607671 32311 init.cc:377] Logged in from keytab as oryx/127.24.71.252@KRBTEST.COM (short username oryx)
I20260811 22:25:57.608305 32311 webserver.cc:460] Webserver started at http://127.24.71.252:45499/ using document root <none> and password file <none>
I20260811 22:25:57.608508 32311 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:57.608558 32311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:57.612362 32311 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:25:57.613570 32532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.613858 32311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 22:25:57.613950 32311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal
uuid: "42a677fee666457192ec1f4407ed69d9"
format_stamp: "Formatted at 2026-08-11 22:25:57 on dist-test-slave-mvvj"
server_key: "2b955b9d9ca072582d60d8949659d91e"
server_key_iv: "ef9ffe703cb4484a3e2bd448d73bc734"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.614058 32311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:57.621734 32311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:57.624054 32311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:57.629760 32311 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.252:34933
I20260811 22:25:57.630573 32311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
I20260811 22:25:57.632709 32590 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:25:57.633821 32590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Bootstrap starting.
I20260811 22:25:57.640002 32589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.252:34933 every 8 connection(s)
W20260811 22:25:57.641927 32508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:57.642350 32508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:57.642439 32508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:57.648072 32508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:57.648152 32508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:57.648262 32508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:25:57.651464 32590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:25:57.651793 32590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Bootstrap complete.
I20260811 22:25:57.652387 32590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } }
I20260811 22:25:57.652516 32590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 42a677fee666457192ec1f4407ed69d9, State: Initialized, Role: LEARNER
I20260811 22:25:57.652634 32590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } }
I20260811 22:25:57.653024 32596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:57.653107 32590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:25:57.653149 32596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:25:57.654302 32508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab
--principal=oryx/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:41421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=45731
--tserver_master_addrs=127.24.71.254:35027,127.24.71.253:35393,127.24.71.252:34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:57.655730 32508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:57.656562 32610 catalog_manager.cc:1259] Loaded cluster ID: c8e20ee46ee141e7bac9731a76061819
I20260811 22:25:57.656622 32610 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: loading cluster ID for follower catalog manager: success
I20260811 22:25:57.657066 32508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:25:57.658669 32610 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: acquiring CA information for follower catalog manager: success
I20260811 22:25:57.660071 32610 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 22:25:57.668004 32612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:57.668004 32615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:57.668015 32613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:57.668381 32508 server_base.cc:1034] running on GCE node
I20260811 22:25:57.668816 32508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:57.669515 32508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:57.670701 32508 hybrid_clock.cc:648] HybridClock initialized: now 1786487157670683 us; error 36 us; skew 500 ppm
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:57.674009 32508 init.cc:377] Logged in from keytab as oryx/127.24.71.193@KRBTEST.COM (short username oryx)
I20260811 22:25:57.675123 32508 webserver.cc:460] Webserver started at http://127.24.71.193:45731/ using document root <none> and password file <none>
I20260811 22:25:57.675767 32508 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:57.675953 32508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:57.680528 32508 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:57.682904 32622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.684473 32508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:57.684633 32508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "35f107625eac4a55b7eb0301a96ff278"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "4bd6e833eae38b0ba8b77f72fc7ecbb0"
server_key_iv: "192e9a34f5392eefd5ac3ccd87fa9f20"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.685240 32508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:57.711050 32508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:57.715040 32508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:57.715337 32508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:57.716114 32508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:57.717206 32508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:57.717286 32508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.717368 32508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:57.717415 32508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.728971 32508 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:41421
I20260811 22:25:57.729036 32749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:41421 every 8 connection(s)
I20260811 22:25:57.730412 32508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:25:57.732039 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 32508
I20260811 22:25:57.732270 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 31903
I20260811 22:25:57.744794 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:46249
--local_ip_for_outbound_sockets=127.24.71.194
--tserver_master_addrs=127.24.71.254:35027
--webserver_port=33603
--webserver_interface=127.24.71.194
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab
--principal=oryx/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.71.254:35027,127.24.71.253:35393
--tserver_master_addrs=127.24.71.254:35027,127.24.71.253:35393,127.24.71.252:34933 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.252@KRBTEST.COM
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.253@KRBTEST.COM
I20260811 22:25:57.754303 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.736563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:54275 (local address 127.24.71.254:35027)
0811 22:25:57.736739 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.736743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.738147 (+ 1404us) server_negotiation.cc:407] Connection header received
0811 22:25:57.739444 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.739451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.739524 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.739585 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.741875 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.742761 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.744754 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.745114 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.745221 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.749132 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.749135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.749137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.749165 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.750931 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.751941 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.751947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.751949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.752057 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.752574 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.752579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.752582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.752854 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.752973 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.753928 (+ 955us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.754111 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:25:57.756520 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.733044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:55267 (local address 127.24.71.253:35393)
0811 22:25:57.736567 (+ 3523us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.736571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.738150 (+ 1579us) server_negotiation.cc:407] Connection header received
0811 22:25:57.739434 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.739438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.739513 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.739571 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.741874 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.742732 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.745855 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.746785 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.746917 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.751208 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.751214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.751220 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.751255 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.753069 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.753647 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.753652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.753654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.753751 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.754076 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.754080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.754082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.754289 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.754384 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.756232 (+ 1848us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.756370 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3406}
I20260811 22:25:57.756476 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.733119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:34933 (local address 127.24.71.193:33907)
0811 22:25:57.737930 (+ 4811us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.738017 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.739131 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.740504 (+ 1373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.740506 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.740775 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.741667 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.741690 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.743076 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.743082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.744606 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.744619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.745414 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.745418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.745660 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.746387 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.746417 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.750307 (+ 3890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.753210 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.753217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.753221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.753593 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.754126 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.754133 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.754137 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.754243 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.754818 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.754822 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.755017 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.755130 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.755308 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4524,"mutex_wait_us":407,"spinlock_wait_cycles":57472,"thread_start_us":691,"threads_started":1}
I20260811 22:25:57.756835 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.733119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.24.71.193:55267)
0811 22:25:57.737930 (+ 4811us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.738017 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.739131 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.740001 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.740013 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.740779 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.741667 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.741690 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.742926 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.742930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.745704 (+ 2774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.745713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.746926 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.746930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.747035 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.748732 (+ 1697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.748750 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.751037 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.753186 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.753189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.753190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.753520 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.753861 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.753865 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.753867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.753966 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.754436 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.754439 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.754542 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.754675 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.755105 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4556,"mutex_wait_us":326,"spinlock_wait_cycles":55296,"thread_start_us":663,"threads_started":1}
I20260811 22:25:57.756475 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.734926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.193:54275)
0811 22:25:57.737934 (+ 3008us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.738017 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.739135 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.740002 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.740015 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.740740 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.741679 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.741694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.742927 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.742933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.744603 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.744615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.745414 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.745418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.745970 (+ 552us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.746493 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.746512 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.748605 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.751144 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.751154 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.751176 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.751660 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.752188 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.752190 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.752192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.752433 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.753105 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.753118 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.753534 (+ 416us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.754950 (+ 1416us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.755173 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2909,"mutex_wait_us":812,"spinlock_wait_cycles":138368,"thread_start_us":128,"threads_started":1}
I20260811 22:25:57.757575 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.735083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:33907 (local address 127.24.71.252:34933)
0811 22:25:57.736871 (+ 1788us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.736884 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.738618 (+ 1734us) server_negotiation.cc:407] Connection header received
0811 22:25:57.739542 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.739548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.739619 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.739704 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.741874 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.742947 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.744763 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.745207 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.745338 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.750484 (+ 5146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.750491 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.750497 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.750536 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.753062 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.753732 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.753740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.753743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.753880 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.754368 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.754374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.754378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.754652 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.754763 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.757167 (+ 2404us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.757341 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1711,"thread_start_us":192,"threads_started":1}
I20260811 22:25:57.758448 32750 heartbeater.cc:347] Connected to a master server at 127.24.71.254:35027
I20260811 22:25:57.758471 32752 heartbeater.cc:347] Connected to a master server at 127.24.71.252:34933
I20260811 22:25:57.758731 32752 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:57.758731 32750 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:57.758761 32753 heartbeater.cc:347] Connected to a master server at 127.24.71.253:35393
I20260811 22:25:57.759227 32753 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:57.759294 32752 heartbeater.cc:511] Master 127.24.71.252:34933 requested a full tablet report, sending...
I20260811 22:25:57.759418 32753 heartbeater.cc:511] Master 127.24.71.253:35393 requested a full tablet report, sending...
I20260811 22:25:57.759871 32750 heartbeater.cc:511] Master 127.24.71.254:35027 requested a full tablet report, sending...
I20260811 22:25:57.760883 31685 ts_manager.cc:194] Re-registered known tserver with Master: 35f107625eac4a55b7eb0301a96ff278 (127.24.71.193:41421)
I20260811 22:25:57.761094 32352 ts_manager.cc:194] Registered new tserver with Master: 35f107625eac4a55b7eb0301a96ff278 (127.24.71.193:41421)
I20260811 22:25:57.761191 32550 ts_manager.cc:194] Registered new tserver with Master: 35f107625eac4a55b7eb0301a96ff278 (127.24.71.193:41421)
I20260811 22:25:57.761695 31685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:54275
I20260811 22:25:57.796799 32197 heartbeater.cc:503] Master 127.24.71.254:35027 was elected leader, sending a full tablet report...
W20260811 22:25:57.872800 32759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:57.873099 32759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:57.873147 32759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:57.877624 32759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:57.877702 32759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:57.877794 32759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:25:57.883383 32759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab
--principal=oryx/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:46249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=33603
--tserver_master_addrs=127.24.71.254:35027,127.24.71.253:35393,127.24.71.252:34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:57.884618 32759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:57.885685 32759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:57.896854 32765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:57.896847 32767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:57.896847 32764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:57.897573 32759 server_base.cc:1034] running on GCE node
I20260811 22:25:57.898020 32759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:57.898733 32759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:57.899912 32759 hybrid_clock.cc:648] HybridClock initialized: now 1786487157899867 us; error 55 us; skew 500 ppm
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:57.902997 32759 init.cc:377] Logged in from keytab as oryx/127.24.71.194@KRBTEST.COM (short username oryx)
I20260811 22:25:57.904079 32759 webserver.cc:460] Webserver started at http://127.24.71.194:33603/ using document root <none> and password file <none>
I20260811 22:25:57.904670 32759 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:57.904779 32759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:57.909336 32759 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 22:25:57.911579 307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.912873 32759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:25:57.913033 32759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "dbc9741b90d5474fa3268cca3bdbcfc3"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "85561d2fb2e8a17243b3161f7390ddbb"
server_key_iv: "6640bff87d17adac5e685df3c327a4bd"
server_key_version: "encryptionkey@0"
I20260811 22:25:57.913594 32759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:57.948443 32759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:57.952039 32759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:57.952258 32759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:57.952905 32759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:57.953974 32759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:57.954048 32759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.954115 32759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:57.954149 32759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:57.964412 32759 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:46249
I20260811 22:25:57.964433 433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:46249 every 8 connection(s)
I20260811 22:25:57.965559 32759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:25:57.966655 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 32759
I20260811 22:25:57.966846 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 32052
I20260811 22:25:57.980934 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:34493
--local_ip_for_outbound_sockets=127.24.71.195
--tserver_master_addrs=127.24.71.254:35027
--webserver_port=36759
--webserver_interface=127.24.71.195
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab
--principal=oryx/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.71.254:35027,127.24.71.253:35393
--tserver_master_addrs=127.24.71.254:35027,127.24.71.253:35393,127.24.71.252:34933 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.253@KRBTEST.COM
Aug 11 22:25:57 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487157, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.252@KRBTEST.COM
I20260811 22:25:57.988729 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.970363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:51579 (local address 127.24.71.254:35027)
0811 22:25:57.970524 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.970531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.972851 (+ 2320us) server_negotiation.cc:407] Connection header received
0811 22:25:57.973047 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.973050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.973120 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.973224 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.975104 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.976033 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.978465 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.978930 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.979042 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.982799 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.982803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.982806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.982843 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.985667 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.986503 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.986507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.986509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.986619 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.987188 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.987192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.987193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.987429 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.987511 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.988404 (+ 893us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.988558 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:25:57.988729 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.973632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:53885 (local address 127.24.71.253:35393)
0811 22:25:57.973779 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.973784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.973800 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:57.973852 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.973855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.973901 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.973985 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.975111 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.976057 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.978464 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.978899 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.979001 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.983559 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.983564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.983567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.983611 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.985646 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.986509 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.986512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.986514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.986734 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.987188 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.987191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.987193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.987451 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.987522 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.988404 (+ 882us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.988558 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:25:57.989854 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.971813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:56757 (local address 127.24.71.252:34933)
0811 22:25:57.976010 (+ 4197us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:57.976016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:57.976033 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:25:57.976078 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:57.976082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:57.976146 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:57.976235 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:57.980386 (+ 4151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.981230 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.982453 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.982811 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.982897 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.985542 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:57.985546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:57.985550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:57.985588 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:57.987429 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.987989 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.987993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.987995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.988085 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:57.988386 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:57.988390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:57.988392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:57.988608 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:57.988689 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:57.989553 (+ 864us) server_negotiation.cc:299] Negotiation successful
0811 22:25:57.989706 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4088}
I20260811 22:25:57.990839 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.970578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.194:51579)
0811 22:25:57.972709 (+ 2131us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.972728 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.972859 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.973449 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.973463 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.974088 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.974914 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.974940 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.976237 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.976240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.978259 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.978276 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.979256 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.979263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.979570 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.980553 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.980568 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.982586 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.985801 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.985808 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.985833 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.986369 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.986888 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.986893 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.986898 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.987066 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.987606 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.987613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.988065 (+ 452us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.989037 (+ 972us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.989520 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2032,"mutex_wait_us":288,"spinlock_wait_cycles":19712,"thread_start_us":232,"threads_started":1}
I20260811 22:25:57.991133 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.972283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:34933 (local address 127.24.71.194:56757)
0811 22:25:57.973241 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.973256 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.973346 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.979721 (+ 6375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.979724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.980038 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.980197 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.980206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.981400 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.981403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.982302 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.982314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.983113 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.983119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.983223 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.983685 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.983699 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.985369 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.987576 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.987580 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.987582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.987858 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.988195 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.988198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.988200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.988283 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.988748 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.988751 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.989333 (+ 582us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.989447 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.989604 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"thread_start_us":157,"threads_started":1}
I20260811 22:25:57.991240 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:57.968623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.24.71.194:53885)
0811 22:25:57.971308 (+ 2685us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:57.971377 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:57.972745 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:57.974131 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:57.974135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:57.974379 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:57.974910 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.974941 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.976233 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.976247 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.978262 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:57.978275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:57.979256 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:57.979260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:57.979560 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:57.980467 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:57.980493 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:57.983371 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:57.985801 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.985808 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.985832 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.986369 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.986845 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:57.986851 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:57.986854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:57.987066 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:57.987606 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:57.987613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:57.988065 (+ 452us) client_negotiation.cc:769] Sending connection context
0811 22:25:57.989005 (+ 940us) client_negotiation.cc:240] Negotiation successful
0811 22:25:57.990141 (+ 1136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2372,"mutex_wait_us":948,"spinlock_wait_cycles":24960,"thread_start_us":483,"threads_started":1}
I20260811 22:25:57.992776 434 heartbeater.cc:347] Connected to a master server at 127.24.71.254:35027
I20260811 22:25:57.993007 436 heartbeater.cc:347] Connected to a master server at 127.24.71.253:35393
I20260811 22:25:57.993103 436 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:57.993105 434 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:57.993260 435 heartbeater.cc:347] Connected to a master server at 127.24.71.252:34933
I20260811 22:25:57.993361 435 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:57.993808 436 heartbeater.cc:511] Master 127.24.71.253:35393 requested a full tablet report, sending...
I20260811 22:25:57.993808 434 heartbeater.cc:511] Master 127.24.71.254:35027 requested a full tablet report, sending...
I20260811 22:25:57.993808 435 heartbeater.cc:511] Master 127.24.71.252:34933 requested a full tablet report, sending...
I20260811 22:25:57.995139 32352 ts_manager.cc:194] Registered new tserver with Master: dbc9741b90d5474fa3268cca3bdbcfc3 (127.24.71.194:46249)
I20260811 22:25:57.995544 32550 ts_manager.cc:194] Registered new tserver with Master: dbc9741b90d5474fa3268cca3bdbcfc3 (127.24.71.194:46249)
I20260811 22:25:57.995602 31685 ts_manager.cc:194] Re-registered known tserver with Master: dbc9741b90d5474fa3268cca3bdbcfc3 (127.24.71.194:46249)
I20260811 22:25:57.996310 31685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:51579
I20260811 22:25:58.100736 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.096054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:34933 (local address 127.0.0.1:40850)
0811 22:25:58.096340 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:58.096358 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:58.096507 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:58.096943 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:58.096946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:58.096970 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:58.097081 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.097087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.098389 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.098394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.099209 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.099220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.100289 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.100293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.100404 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.100434 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:25:58.100507 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:25:58.100588 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":101,"threads_started":1}
I20260811 22:25:58.100782 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.096173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.24.71.252:34933)
0811 22:25:58.096366 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.096373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.096473 (+ 100us) server_negotiation.cc:407] Connection header received
0811 22:25:58.096606 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.096612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.096699 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.096848 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:58.097229 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.098203 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.099358 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.100127 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.100252 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.100469 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.100528 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.100630 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:25:58.101711 32561 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 22:25:58.102161 32596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9918892880914ddf8b203bc656c08730. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:58.102320 32596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: This master's current role is: LEARNER
W20260811 22:25:58.134771 442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:25:58.135063 442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:25:58.135109 442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:25:58.139247 442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:25:58.139322 442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:25:58.139389 442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:25:58.145169 442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab
--principal=oryx/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:34493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=36759
--tserver_master_addrs=127.24.71.254:35027,127.24.71.253:35393,127.24.71.252:34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:25:58.146337 442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:25:58.147426 442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:25:58.158089 451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:58.158180 442 server_base.cc:1034] running on GCE node
W20260811 22:25:58.158068 449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:25:58.158063 448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:25:58.159024 442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:25:58.159719 442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:25:58.160879 442 hybrid_clock.cc:648] HybridClock initialized: now 1786487158160850 us; error 37 us; skew 500 ppm
Aug 11 22:25:58 dist-test-slave-mvvj krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487158, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:25:58.164168 442 init.cc:377] Logged in from keytab as oryx/127.24.71.195@KRBTEST.COM (short username oryx)
I20260811 22:25:58.165190 442 webserver.cc:460] Webserver started at http://127.24.71.195:36759/ using document root <none> and password file <none>
I20260811 22:25:58.165804 442 fs_manager.cc:359] Metadata directory not provided
I20260811 22:25:58.165921 442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:25:58.170060 442 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:25:58.172763 459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:58.174086 442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:25:58.174242 442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "fceaadf920814f1ab7560a10d13a8385"
format_stamp: "Formatted at 2026-08-11 22:25:56 on dist-test-slave-mvvj"
server_key: "32e9c095e3fb2da4590f7bc8a441e726"
server_key_iv: "b6359c16137128aa8f5414ea7913e24f"
server_key_version: "encryptionkey@0"
I20260811 22:25:58.174817 442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:25:58.192714 442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:25:58.196552 442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:25:58.196768 442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:25:58.197443 442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:25:58.198385 442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:25:58.198434 442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:58.198501 442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:25:58.198534 442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:25:58.207993 442 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:34493
I20260811 22:25:58.208056 583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:34493 every 8 connection(s)
I20260811 22:25:58.209103 442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:25:58.211326 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 442
Aug 11 22:25:58 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487158, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
Aug 11 22:25:58 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487158, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.252@KRBTEST.COM
Aug 11 22:25:58 dist-test-slave-mvvj krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487158, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.253@KRBTEST.COM
I20260811 22:25:58.230443 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.213567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:45111 (local address 127.24.71.253:35393)
0811 22:25:58.213829 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.213833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.213851 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:25:58.213919 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.213923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.213987 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.214075 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.216280 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.217174 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.219475 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.219852 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.219937 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.225802 (+ 5865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.225805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.225808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.225835 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.227707 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.228279 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.228286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.228291 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.228384 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.228857 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.228863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.228868 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.229130 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.229222 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.230085 (+ 863us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.230232 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260811 22:25:58.230683 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.211118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:48113 (local address 127.24.71.252:34933)
0811 22:25:58.211307 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.211312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.212302 (+ 990us) server_negotiation.cc:407] Connection header received
0811 22:25:58.214116 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.214119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.214173 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.214228 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.216279 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.217251 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.219462 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.219796 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.219901 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.225496 (+ 5595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.225500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.225503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.225534 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.227283 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.228419 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.228423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.228425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.228535 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.229516 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.229520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.229521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.229757 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.229861 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.230357 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.230487 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:25:58.232650 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.213269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:52407 (local address 127.24.71.254:35027)
0811 22:25:58.213427 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.213432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.213533 (+ 101us) server_negotiation.cc:407] Connection header received
0811 22:25:58.214094 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.214098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.214154 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.214247 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.216280 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.217266 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.219476 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.219857 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.219937 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.225649 (+ 5712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.225653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.225656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.225684 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.228245 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.230897 (+ 2652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.230901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.230903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.231015 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.231354 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.231358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.231361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.231614 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.231729 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.232309 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.232459 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:25:58.232823 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.212610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.24.71.195:45111)
0811 22:25:58.212888 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:58.212902 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:58.213724 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:58.214279 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:58.214288 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:58.214959 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:58.216087 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.216110 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.217401 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.217405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.219317 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.219330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.220158 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.220162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.220535 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.223862 (+ 3327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:58.223881 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:58.225629 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:58.227879 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:58.227884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:58.227886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:58.228151 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:58.228521 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:58.228525 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:58.228527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:58.228738 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:58.229307 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:58.229314 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:58.229732 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 22:25:58.230744 (+ 1012us) client_negotiation.cc:240] Negotiation successful
0811 22:25:58.231239 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":578,"spinlock_wait_cycles":43776,"thread_start_us":125,"threads_started":1}
I20260811 22:25:58.233345 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.211544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:34933 (local address 127.24.71.195:48113)
0811 22:25:58.212121 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:58.212183 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:58.213708 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:58.214426 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:58.214430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:58.215056 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:58.216084 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.216108 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.217434 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.217439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.219319 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.219331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.220163 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.220167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.220541 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.221676 (+ 1135us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:58.221699 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:58.225314 (+ 3615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:58.227464 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:58.227496 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:58.227530 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:58.227976 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:58.229284 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:58.229287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:58.229289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:58.229387 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:58.229913 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:58.229917 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:58.230047 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 22:25:58.231055 (+ 1008us) client_negotiation.cc:240] Negotiation successful
0811 22:25:58.231924 (+ 869us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":1159,"spinlock_wait_cycles":45952,"thread_start_us":122,"threads_started":1}
I20260811 22:25:58.233459 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.213143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.195:52407)
0811 22:25:58.213441 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:58.213456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:58.213908 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:58.214388 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:58.214391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:58.214959 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:25:58.216086 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.216110 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.217435 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.217437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.219317 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.219330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.220158 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.220162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.220522 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.221469 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:25:58.221493 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:25:58.225484 (+ 3991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:25:58.228367 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:58.228370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:58.228374 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:58.228612 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:58.231145 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:25:58.231148 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:25:58.231150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:25:58.231245 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:25:58.231781 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:25:58.231786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:25:58.231927 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 22:25:58.232825 (+ 898us) client_negotiation.cc:240] Negotiation successful
0811 22:25:58.232981 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":713,"spinlock_wait_cycles":62848,"thread_start_us":123,"threads_started":1}
I20260811 22:25:58.234436 586 heartbeater.cc:347] Connected to a master server at 127.24.71.253:35393
I20260811 22:25:58.234716 586 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:58.235059 584 heartbeater.cc:347] Connected to a master server at 127.24.71.254:35027
I20260811 22:25:58.235154 584 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:58.235287 599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.215769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.24.71.253:35393)
0811 22:25:58.221368 (+ 5599us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.221374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.222624 (+ 1250us) server_negotiation.cc:407] Connection header received
0811 22:25:58.222740 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.222744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.222799 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.222904 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.223662 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.224533 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.226381 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.226805 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.226913 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.228617 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.228622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.228625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.228650 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.231422 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.232168 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.232171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.232173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.232265 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.233092 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.233095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.233097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.233322 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.233461 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.234947 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.235111 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5497,"thread_start_us":121,"threads_started":1}
I20260811 22:25:58.235425 586 heartbeater.cc:511] Master 127.24.71.253:35393 requested a full tablet report, sending...
I20260811 22:25:58.235426 584 heartbeater.cc:511] Master 127.24.71.254:35027 requested a full tablet report, sending...
I20260811 22:25:58.235955 585 heartbeater.cc:347] Connected to a master server at 127.24.71.252:34933
I20260811 22:25:58.236050 585 heartbeater.cc:464] Registering TS with master...
I20260811 22:25:58.236218 585 heartbeater.cc:511] Master 127.24.71.252:34933 requested a full tablet report, sending...
I20260811 22:25:58.236770 32352 ts_manager.cc:194] Registered new tserver with Master: fceaadf920814f1ab7560a10d13a8385 (127.24.71.195:34493)
I20260811 22:25:58.236886 31685 ts_manager.cc:194] Re-registered known tserver with Master: fceaadf920814f1ab7560a10d13a8385 (127.24.71.195:34493)
I20260811 22:25:58.236963 32550 ts_manager.cc:194] Registered new tserver with Master: fceaadf920814f1ab7560a10d13a8385 (127.24.71.195:34493)
I20260811 22:25:58.237673 31685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:52407
I20260811 22:25:58.239351 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.213549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.24.71.252:34933)
0811 22:25:58.213859 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.213865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.216612 (+ 2747us) server_negotiation.cc:407] Connection header received
0811 22:25:58.216757 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.216760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.216809 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.216900 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.219114 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.219968 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.221204 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.221596 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.221713 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.224410 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.224415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.224418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.224450 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.234655 (+ 10205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.237705 (+ 3050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.237708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.237710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.237829 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.238173 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.238176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.238178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.238443 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.238575 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.238997 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.239165 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":115,"threads_started":1}
I20260811 22:25:58.240017 597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.214957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54886 (local address 127.24.71.254:35027)
0811 22:25:58.222011 (+ 7054us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.222016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.222030 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:25:58.222073 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.222076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.222127 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.222231 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.223910 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.224699 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.228645 (+ 3946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.229055 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.229160 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.231227 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.231232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.231235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.231261 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.235087 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.237959 (+ 2872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.237963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.237965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.238056 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.238377 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.238381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.238383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.238586 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.238742 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.239170 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.239308 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6966,"thread_start_us":78,"threads_started":1}
I20260811 22:25:58.243405 31685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:25:58.246927 31685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:25:58.250106 32308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: NON_VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: true } } }
I20260811 22:25:58.250293 32308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:58.254245 32308 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: attempting to promote NON_VOTER 42a677fee666457192ec1f4407ed69d9 to VOTER
I20260811 22:25:58.254586 32308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } }
I20260811 22:25:58.256954 32561 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 LEARNER]: Refusing update from remote peer 9918892880914ddf8b203bc656c08730: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:25:58.257436 603 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:25:58.257520 32363 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Refusing update from remote peer 9918892880914ddf8b203bc656c08730: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:25:58.258037 603 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:25:58.260375 604 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 42a677fee666457192ec1f4407ed69d9 (127.24.71.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:25:58.261067 32561 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 42a677fee666457192ec1f4407ed69d9 (127.24.71.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:25:58.261844 604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:25:58.261981 604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:58.262135 604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:25:58.262236 604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:58.262518 604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:25:58.262635 604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730 [sys.catalog]: This master's current role is: LEADER
I20260811 22:25:58.263103 32596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:25:58.263212 32596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:25:58.263522 32363 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 42a677fee666457192ec1f4407ed69d9 (127.24.71.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:25:58.267041 613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9918892880914ddf8b203bc656c08730" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:25:58.267199 613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:25:58.272720 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.265445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:41421 (local address 127.0.0.1:58830)
0811 22:25:58.266231 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:58.266271 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:58.266503 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:58.267637 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:58.267642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:58.267668 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:58.267792 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.267800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.269823 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.269827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.271178 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.271190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.272251 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.272255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.272357 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.272387 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:25:58.272460 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:25:58.272552 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":28}
I20260811 22:25:58.272758 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.266578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.24.71.193:41421)
0811 22:25:58.267016 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.267026 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.267053 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:25:58.267147 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.267153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.267452 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.268242 (+ 790us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:58.268412 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.269672 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.271285 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.272120 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.272232 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.272420 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.272490 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.272594 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":91,"threads_started":1}
I20260811 22:25:58.273236 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.267556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46249 (local address 127.0.0.1:37794)
0811 22:25:58.268004 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:58.268020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:58.268118 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:58.268601 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:58.268604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:58.268623 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:58.268738 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.268746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.269693 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.269698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.271047 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.271057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.272490 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.272494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.272586 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.272609 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:25:58.272681 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:25:58.272745 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":66,"thread_start_us":101,"threads_started":1}
I20260811 22:25:58.273394 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.265755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:34493 (local address 127.0.0.1:59554)
0811 22:25:58.267791 (+ 2036us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:58.267807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:58.268013 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:58.268749 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:58.268752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:58.268771 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:58.268944 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.268951 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.270524 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.270528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.271789 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.271800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.272967 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.272971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.273067 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.273091 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:25:58.273180 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:25:58.273257 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1968,"thread_start_us":120,"threads_started":1}
I20260811 22:25:58.273480 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.267135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.24.71.195:34493)
0811 22:25:58.267622 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.267635 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.267932 (+ 297us) server_negotiation.cc:407] Connection header received
0811 22:25:58.268194 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.268201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.268418 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.268557 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:58.269098 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.270379 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.271940 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.272826 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.272952 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.273103 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.273211 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.273309 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":121,"threads_started":1}
I20260811 22:25:58.274150 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.267716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37794 (local address 127.24.71.194:46249)
0811 22:25:58.268185 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.268199 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.268222 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:25:58.268296 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.268300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.268460 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.268628 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:58.268860 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.269572 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.271170 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.272118 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.272232 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.272420 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.273850 (+ 1430us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.274011 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":155,"threads_started":1}
I20260811 22:25:58.276530 360 tablet_service.cc:1467] Processing CreateTablet for tablet 7489d3fd32674627b94bb694e6244c06 (DEFAULT_TABLE table=test-table [id=725ec724bff144d6827de777eeba9bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:58.276537 513 tablet_service.cc:1467] Processing CreateTablet for tablet 7489d3fd32674627b94bb694e6244c06 (DEFAULT_TABLE table=test-table [id=725ec724bff144d6827de777eeba9bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:58.276535 32677 tablet_service.cc:1467] Processing CreateTablet for tablet 7489d3fd32674627b94bb694e6244c06 (DEFAULT_TABLE table=test-table [id=725ec724bff144d6827de777eeba9bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:25:58.277511 360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7489d3fd32674627b94bb694e6244c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:58.277655 32677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7489d3fd32674627b94bb694e6244c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:58.277655 513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7489d3fd32674627b94bb694e6244c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:25:58.285810 620 tablet_bootstrap.cc:492] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: Bootstrap starting.
I20260811 22:25:58.286989 621 tablet_bootstrap.cc:492] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: Bootstrap starting.
I20260811 22:25:58.286989 622 tablet_bootstrap.cc:492] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: Bootstrap starting.
I20260811 22:25:58.288532 620 tablet_bootstrap.cc:654] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:58.289420 622 tablet_bootstrap.cc:654] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:58.289422 621 tablet_bootstrap.cc:654] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: Neither blocks nor log segments found. Creating new log.
I20260811 22:25:58.289429 620 log.cc:824] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:58.290328 621 log.cc:824] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:58.290338 622 log.cc:824] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: Log is configured to *not* fsync() on all Append() calls
I20260811 22:25:58.292004 620 tablet_bootstrap.cc:492] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: No bootstrap required, opened a new log
I20260811 22:25:58.292225 620 ts_tablet_manager.cc:1397] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:25:58.292510 622 tablet_bootstrap.cc:492] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: No bootstrap required, opened a new log
I20260811 22:25:58.292523 621 tablet_bootstrap.cc:492] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: No bootstrap required, opened a new log
I20260811 22:25:58.292704 622 ts_tablet_manager.cc:1397] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260811 22:25:58.292742 621 ts_tablet_manager.cc:1397] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:25:58.296736 621 raft_consensus.cc:348] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } }
I20260811 22:25:58.297034 621 raft_consensus.cc:374] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:58.297096 621 raft_consensus.cc:729] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f107625eac4a55b7eb0301a96ff278, State: Initialized, Role: FOLLOWER
I20260811 22:25:58.297426 622 raft_consensus.cc:348] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } }
I20260811 22:25:58.297425 620 raft_consensus.cc:348] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } }
I20260811 22:25:58.297674 622 raft_consensus.cc:374] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:58.297675 620 raft_consensus.cc:374] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:25:58.297734 620 raft_consensus.cc:729] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbc9741b90d5474fa3268cca3bdbcfc3, State: Initialized, Role: FOLLOWER
I20260811 22:25:58.297744 622 raft_consensus.cc:729] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fceaadf920814f1ab7560a10d13a8385, State: Initialized, Role: FOLLOWER
I20260811 22:25:58.297632 621 consensus_queue.cc:260] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } }
I20260811 22:25:58.298166 620 consensus_queue.cc:260] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } }
I20260811 22:25:58.298166 622 consensus_queue.cc:260] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } }
I20260811 22:25:58.298563 32750 heartbeater.cc:503] Master 127.24.71.254:35027 was elected leader, sending a full tablet report...
I20260811 22:25:58.298879 621 ts_tablet_manager.cc:1428] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:25:58.299163 622 ts_tablet_manager.cc:1428] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260811 22:25:58.299160 620 ts_tablet_manager.cc:1428] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:25:58.299497 584 heartbeater.cc:503] Master 127.24.71.254:35027 was elected leader, sending a full tablet report...
I20260811 22:25:58.305070 434 heartbeater.cc:503] Master 127.24.71.254:35027 was elected leader, sending a full tablet report...
I20260811 22:25:58.396248 626 raft_consensus.cc:482] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:25:58.396520 626 raft_consensus.cc:504] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } }
I20260811 22:25:58.398121 626 leader_election.cc:288] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbc9741b90d5474fa3268cca3bdbcfc3 (127.24.71.194:46249), fceaadf920814f1ab7560a10d13a8385 (127.24.71.195:34493)
I20260811 22:25:58.404119 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.398553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:37581 (local address 127.24.71.195:34493)
0811 22:25:58.398708 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.398713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.399055 (+ 342us) server_negotiation.cc:407] Connection header received
0811 22:25:58.399194 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.399200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.399291 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.399392 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:58.400052 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.400813 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.402676 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.403298 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.403531 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.403592 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.403884 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.403973 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:25:58.404119 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.398498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:32917 (local address 127.24.71.194:46249)
0811 22:25:58.398666 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.398671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.398969 (+ 298us) server_negotiation.cc:407] Connection header received
0811 22:25:58.399179 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.399182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.399245 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.399300 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:25:58.400000 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.400813 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.402676 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.403299 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.403443 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.403517 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.403918 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.403990 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:25:58.404222 631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.398453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46249 (local address 127.24.71.193:32917)
0811 22:25:58.398869 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:58.398882 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:58.399024 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:58.399507 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:58.399510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:58.399530 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:58.399771 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.399782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.400960 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.400964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.402508 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.402519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.403523 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.403527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.403690 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.403747 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 22:25:58.403850 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:25:58.404027 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":336,"spinlock_wait_cycles":6272,"thread_start_us":160,"threads_started":1}
I20260811 22:25:58.404233 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.398404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:34493 (local address 127.24.71.193:37581)
0811 22:25:58.398857 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:58.398875 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:58.399025 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:25:58.399545 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:25:58.399550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:25:58.399587 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:25:58.399878 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.399915 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.400974 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.400976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.402505 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:25:58.402519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.403522 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.403527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:25:58.403687 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.403743 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 22:25:58.403850 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:25:58.404027 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":214,"spinlock_wait_cycles":12928,"thread_start_us":143,"threads_started":1}
I20260811 22:25:58.405154 534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7489d3fd32674627b94bb694e6244c06" candidate_uuid: "35f107625eac4a55b7eb0301a96ff278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fceaadf920814f1ab7560a10d13a8385" is_pre_election: true
I20260811 22:25:58.405253 383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7489d3fd32674627b94bb694e6244c06" candidate_uuid: "35f107625eac4a55b7eb0301a96ff278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" is_pre_election: true
I20260811 22:25:58.405551 534 raft_consensus.cc:2454] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f107625eac4a55b7eb0301a96ff278 in term 0.
I20260811 22:25:58.405586 383 raft_consensus.cc:2454] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f107625eac4a55b7eb0301a96ff278 in term 0.
I20260811 22:25:58.406109 32623 leader_election.cc:302] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 35f107625eac4a55b7eb0301a96ff278, dbc9741b90d5474fa3268cca3bdbcfc3, fceaadf920814f1ab7560a10d13a8385; no voters:
I20260811 22:25:58.406442 626 raft_consensus.cc:2781] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:25:58.406523 626 raft_consensus.cc:482] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:25:58.406555 626 raft_consensus.cc:3037] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:58.407680 626 raft_consensus.cc:504] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } }
I20260811 22:25:58.408104 626 leader_election.cc:288] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [CANDIDATE]: Term 1 election: Requested vote from peers dbc9741b90d5474fa3268cca3bdbcfc3 (127.24.71.194:46249), fceaadf920814f1ab7560a10d13a8385 (127.24.71.195:34493)
I20260811 22:25:58.408468 383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7489d3fd32674627b94bb694e6244c06" candidate_uuid: "35f107625eac4a55b7eb0301a96ff278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3"
I20260811 22:25:58.408475 534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7489d3fd32674627b94bb694e6244c06" candidate_uuid: "35f107625eac4a55b7eb0301a96ff278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fceaadf920814f1ab7560a10d13a8385"
I20260811 22:25:58.408613 383 raft_consensus.cc:3037] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:58.408612 534 raft_consensus.cc:3037] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:25:58.409763 534 raft_consensus.cc:2454] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f107625eac4a55b7eb0301a96ff278 in term 1.
I20260811 22:25:58.409762 383 raft_consensus.cc:2454] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f107625eac4a55b7eb0301a96ff278 in term 1.
I20260811 22:25:58.410070 32625 leader_election.cc:302] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f107625eac4a55b7eb0301a96ff278, fceaadf920814f1ab7560a10d13a8385; no voters:
I20260811 22:25:58.410282 626 raft_consensus.cc:2781] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:25:58.410542 626 raft_consensus.cc:686] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 1 LEADER]: Becoming Leader. State: Replica: 35f107625eac4a55b7eb0301a96ff278, State: Running, Role: LEADER
I20260811 22:25:58.410969 626 consensus_queue.cc:237] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } }
I20260811 22:25:58.414862 31685 catalog_manager.cc:5495] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35f107625eac4a55b7eb0301a96ff278 (127.24.71.193). New cstate: current_term: 1 leader_uuid: "35f107625eac4a55b7eb0301a96ff278" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } health_report { overall_health: UNKNOWN } } }
W20260811 22:25:58.461104 587 tablet.cc:2369] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:58.463006 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.459217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58836 (local address 127.24.71.193:41421)
0811 22:25:58.459386 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.459392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.459408 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:25:58.459510 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.459512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.459569 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.459688 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:25:58.460025 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.460541 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.461501 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.461724 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.461835 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.462183 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:25:58.462600 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:25:58.462741 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.462783 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.462849 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 22:25:58.467944 437 tablet.cc:2369] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:25:58.469767 534 raft_consensus.cc:1261] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [term 1 FOLLOWER]: Refusing update from remote peer 35f107625eac4a55b7eb0301a96ff278: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:58.469766 383 raft_consensus.cc:1261] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 1 FOLLOWER]: Refusing update from remote peer 35f107625eac4a55b7eb0301a96ff278: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:25:58.470690 632 consensus_queue.cc:1035] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:58.470963 626 consensus_queue.cc:1035] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:25:58.490672 31686 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:58.490883 31686 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:25:58.496290 31686 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9918892880914ddf8b203bc656c08730: Sending DeleteTablet for 3 replicas of tablet 7489d3fd32674627b94bb694e6244c06
I20260811 22:25:58.497112 360 tablet_service.cc:1514] Processing DeleteTablet for tablet 7489d3fd32674627b94bb694e6244c06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:58 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:37794
I20260811 22:25:58.497632 646 tablet_replica.cc:330] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: stopping tablet replica
I20260811 22:25:58.498047 646 raft_consensus.cc:2229] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:58.498291 32677 tablet_service.cc:1514] Processing DeleteTablet for tablet 7489d3fd32674627b94bb694e6244c06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:58 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58830
I20260811 22:25:58.498445 646 raft_consensus.cc:2258] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:58.498634 513 tablet_service.cc:1514] Processing DeleteTablet for tablet 7489d3fd32674627b94bb694e6244c06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:25:58 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:59554
I20260811 22:25:58.499222 648 tablet_replica.cc:330] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: stopping tablet replica
I20260811 22:25:58.499589 648 raft_consensus.cc:2229] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:25:58.500015 648 raft_consensus.cc:2258] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:25:58.501046 646 ts_tablet_manager.cc:1905] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:58.501785 648 ts_tablet_manager.cc:1905] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:58.502630 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 31655
I20260811 22:25:58.507431 646 ts_tablet_manager.cc:1918] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:58.507630 646 log.cc:1196] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/7489d3fd32674627b94bb694e6244c06
I20260811 22:25:58.508097 646 ts_tablet_manager.cc:1939] T 7489d3fd32674627b94bb694e6244c06 P dbc9741b90d5474fa3268cca3bdbcfc3: Deleting consensus metadata
I20260811 22:25:58.508265 648 ts_tablet_manager.cc:1918] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:58.508420 648 log.cc:1196] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/7489d3fd32674627b94bb694e6244c06
I20260811 22:25:58.508810 648 ts_tablet_manager.cc:1939] T 7489d3fd32674627b94bb694e6244c06 P fceaadf920814f1ab7560a10d13a8385: Deleting consensus metadata
I20260811 22:25:58.514549 647 tablet_replica.cc:330] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: stopping tablet replica
I20260811 22:25:58.515144 647 raft_consensus.cc:2229] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 1 LEADER]: Raft consensus shutting down.
W20260811 22:25:58.515620 309 connection.cc:535] server connection from 127.0.0.1:37794 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:25:58.515758 647 raft_consensus.cc:2258] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 22:25:58.516180 462 connection.cc:535] server connection from 127.0.0.1:59554 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:25:58.517874 647 ts_tablet_manager.cc:1905] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:25:58.521466 647 ts_tablet_manager.cc:1918] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:25:58.521596 647 log.cc:1196] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/7489d3fd32674627b94bb694e6244c06
I20260811 22:25:58.521978 647 ts_tablet_manager.cc:1939] T 7489d3fd32674627b94bb694e6244c06 P 35f107625eac4a55b7eb0301a96ff278: Deleting consensus metadata
W20260811 22:25:58.523972 32625 connection.cc:441] server connection from 127.0.0.1:58830 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58830 (request call id 1) could send its response
I20260811 22:25:58.532761 599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.501204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.24.71.253:35393)
0811 22:25:58.501326 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.501330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.514098 (+ 12768us) server_negotiation.cc:407] Connection header received
0811 22:25:58.514333 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.514337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.514409 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.514568 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.517192 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.518017 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.519253 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.519645 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.519743 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.528506 (+ 8763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.528510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.528514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.528547 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.530570 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.531118 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.531123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.531125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.531219 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.531546 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.531549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.531551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.531784 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.532047 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.532431 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.532574 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 22:25:58.534255 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.521137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51686 (local address 127.24.71.253:35393)
0811 22:25:58.521273 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.521279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.523724 (+ 2445us) server_negotiation.cc:407] Connection header received
0811 22:25:58.523769 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.523772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.523817 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.524283 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.524970 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.525616 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.526980 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.527402 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.527554 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.529386 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.529391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.529396 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.529442 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.531567 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.532220 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.532224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.532226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.532333 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.532825 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.532829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.532831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.533041 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.533155 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.533955 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.534088 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:25:58.534588 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.519735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40868 (local address 127.24.71.252:34933)
0811 22:25:58.519903 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.519909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.521510 (+ 1601us) server_negotiation.cc:407] Connection header received
0811 22:25:58.521676 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.521680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.521740 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.521785 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.523139 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.524010 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.525320 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.525647 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.525748 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.528939 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.528946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.528949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.528991 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.531493 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.532220 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.532224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.532226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.532452 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.532804 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.532807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.532809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.533041 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.533163 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.533955 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.534384 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:25:58.542934 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:58.500993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.24.71.252:34933)
0811 22:25:58.501115 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:25:58.501120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:25:58.533394 (+ 32274us) server_negotiation.cc:407] Connection header received
0811 22:25:58.533588 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:25:58.533591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:25:58.533645 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:25:58.533702 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:25:58.534571 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.535471 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.536923 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:25:58.537169 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:25:58.537268 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:25:58.538948 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:25:58.538951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:25:58.538954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:25:58.538979 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:25:58.540951 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.541503 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.541508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.541514 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.541614 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:25:58.541950 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:25:58.541953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:25:58.541955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:25:58.542172 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:25:58.542334 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:25:58.542581 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 22:25:58.542719 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:25:59.486037 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.485021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.193:59729)
0811 22:25:59.485797 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:59.485916 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":106,"threads_started":1}
I20260811 22:25:59.486291 660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.485734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.195:39787)
0811 22:25:59.486072 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:59.486169 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":81,"threads_started":1}
W20260811 22:25:59.486521 32750 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:35027 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
I20260811 22:25:59.486781 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.486329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.194:33171)
0811 22:25:59.486594 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:59.486681 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":64,"threads_started":1}
W20260811 22:25:59.487063 584 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:35027 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
W20260811 22:25:59.487087 434 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:35027 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
I20260811 22:25:59.997556 662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9918892880914ddf8b203bc656c08730)
I20260811 22:25:59.997620 663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9918892880914ddf8b203bc656c08730)
I20260811 22:25:59.997836 662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } }
I20260811 22:25:59.997836 663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } }
I20260811 22:25:59.998930 663 leader_election.cc:288] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027), 42a677fee666457192ec1f4407ed69d9 (127.24.71.252:34933)
I20260811 22:25:59.998962 662 leader_election.cc:288] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027), 8e796f4ce0654b669a4ed5f8ace642f8 (127.24.71.253:35393)
I20260811 22:25:59.999790 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.999279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55290)
0811 22:25:59.999560 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:59.999665 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
W20260811 22:26:00.000016 32533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
I20260811 22:26:00.000161 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.999215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55282)
0811 22:25:59.999871 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.000046 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":130,"threads_started":1}
W20260811 22:26:00.000347 32333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
I20260811 22:26:00.001294 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:00.001007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55312)
0811 22:26:00.001164 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.001220 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 22:26:00.001452 32333 leader_election.cc:334] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027): Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
I20260811 22:26:00.002799 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:00.001007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55296)
0811 22:26:00.001208 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.002709 (+ 1501us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260811 22:26:00.002961 32533 leader_election.cc:334] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027): Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
I20260811 22:26:00.005775 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.999942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51694 (local address 127.24.71.253:35393)
0811 22:26:00.000388 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:00.000393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:00.000411 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:26:00.000579 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:00.000587 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:00.000653 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:00.000756 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:00.001187 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:00.002397 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:00.004243 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:00.004924 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:00.005053 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:00.005126 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:00.005526 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 22:26:00.005614 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":102,"threads_started":1}
I20260811 22:26:00.005795 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.999626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.24.71.252:34933)
0811 22:26:00.000757 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:00.000762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:00.000777 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:00.000835 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:00.000838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:00.000890 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:00.000972 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:00.001343 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:00.002451 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:00.004243 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:00.004886 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:00.005053 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:00.005126 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:00.005526 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 22:26:00.005614 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":262,"server-negotiator.queue_time_us":556,"thread_start_us":55,"threads_started":1}
I20260811 22:26:00.005756 668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.999597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:35393 (local address 127.0.0.1:51694)
0811 22:26:00.000048 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.000062 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:00.000376 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:00.000887 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:00.000890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:00.000907 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:00.001055 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:00.001063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:00.002559 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:00.002563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:00.004065 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:00.004077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:00.005096 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:00.005101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:00.005227 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:00.005343 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 22:26:00.005450 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:26:00.005552 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":4,"thread_start_us":68,"threads_started":1}
I20260811 22:26:00.005774 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:25:59.999192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:34933 (local address 127.0.0.1:40872)
0811 22:25:59.999582 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 22:25:59.999597 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:25:59.999720 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:00.001055 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:00.001058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:00.001076 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:00.001187 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:00.001193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:00.002582 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:00.002585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:00.004060 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:00.004077 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:00.005096 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:00.005101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:00.005223 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:00.005343 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 22:26:00.005450 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:26:00.005546 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":62,"threads_started":1}
I20260811 22:26:00.006570 32363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42a677fee666457192ec1f4407ed69d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" is_pre_election: true
I20260811 22:26:00.006879 32363 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42a677fee666457192ec1f4407ed69d9 in term 1.
I20260811 22:26:00.006891 32561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "42a677fee666457192ec1f4407ed69d9" is_pre_election: true
I20260811 22:26:00.007138 32561 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e796f4ce0654b669a4ed5f8ace642f8 in term 1.
I20260811 22:26:00.007355 32533 leader_election.cc:302] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 42a677fee666457192ec1f4407ed69d9, 8e796f4ce0654b669a4ed5f8ace642f8; no voters: 9918892880914ddf8b203bc656c08730
I20260811 22:26:00.007542 32335 leader_election.cc:302] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 42a677fee666457192ec1f4407ed69d9, 8e796f4ce0654b669a4ed5f8ace642f8; no voters: 9918892880914ddf8b203bc656c08730
I20260811 22:26:00.007622 662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:26:00.007682 662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9918892880914ddf8b203bc656c08730)
I20260811 22:26:00.007720 662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:26:00.007958 663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:26:00.008029 663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9918892880914ddf8b203bc656c08730)
I20260811 22:26:00.008085 663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:26:00.008953 662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } }
I20260811 22:26:00.009124 663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } }
I20260811 22:26:00.009449 662 leader_election.cc:288] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 2 election: Requested vote from peers 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027), 8e796f4ce0654b669a4ed5f8ace642f8 (127.24.71.253:35393)
I20260811 22:26:00.009567 663 leader_election.cc:288] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [CANDIDATE]: Term 2 election: Requested vote from peers 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027), 42a677fee666457192ec1f4407ed69d9 (127.24.71.252:34933)
I20260811 22:26:00.009840 668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:00.009577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55330)
0811 22:26:00.009684 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.009750 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 22:26:00.010001 32363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42a677fee666457192ec1f4407ed69d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8e796f4ce0654b669a4ed5f8ace642f8"
I20260811 22:26:00.010032 32561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "42a677fee666457192ec1f4407ed69d9"
I20260811 22:26:00.010191 32363 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 42a677fee666457192ec1f4407ed69d9 in current term 2: Already voted for candidate 8e796f4ce0654b669a4ed5f8ace642f8 in this term.
I20260811 22:26:00.010262 32561 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e796f4ce0654b669a4ed5f8ace642f8 in current term 2: Already voted for candidate 42a677fee666457192ec1f4407ed69d9 in this term.
I20260811 22:26:00.010609 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:00.009673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55342)
0811 22:26:00.010456 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.010528 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 22:26:00.011241 668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:00.011018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55348)
0811 22:26:00.011130 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.011180 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 22:26:00.011289 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:00.011078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55362)
0811 22:26:00.011168 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.011223 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 22:26:00.011390 32333 leader_election.cc:334] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027): Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
W20260811 22:26:00.011415 32533 leader_election.cc:334] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027): Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
I20260811 22:26:00.011461 32333 leader_election.cc:302] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e796f4ce0654b669a4ed5f8ace642f8; no voters: 42a677fee666457192ec1f4407ed69d9, 9918892880914ddf8b203bc656c08730
I20260811 22:26:00.011497 32533 leader_election.cc:302] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 42a677fee666457192ec1f4407ed69d9; no voters: 8e796f4ce0654b669a4ed5f8ace642f8, 9918892880914ddf8b203bc656c08730
I20260811 22:26:00.011633 663 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 22:26:00.011767 662 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 22:26:00.487982 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:00.487176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.193:41153)
0811 22:26:00.487752 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.487850 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":130,"threads_started":1}
I20260811 22:26:00.488389 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:00.487606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.195:37135)
0811 22:26:00.488046 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.488281 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":86,"threads_started":1}
I20260811 22:26:00.488965 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:00.487922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.194:48089)
0811 22:26:00.488788 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:00.488865 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":76,"threads_started":1}
I20260811 22:26:01.489537 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.488755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.193:58685)
0811 22:26:01.489093 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.489203 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":118,"threads_started":1}
I20260811 22:26:01.490226 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.489205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.195:43821)
0811 22:26:01.490067 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.490135 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":72,"threads_started":1}
I20260811 22:26:01.490559 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.489701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.194:49351)
0811 22:26:01.490392 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.490458 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":82,"threads_started":1}
I20260811 22:26:01.521447 679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:26:01.521606 679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } }
I20260811 22:26:01.522050 679 leader_election.cc:288] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027), 8e796f4ce0654b669a4ed5f8ace642f8 (127.24.71.253:35393)
I20260811 22:26:01.522789 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.522322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55592)
0811 22:26:01.522621 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.522693 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":138,"threads_started":1}
I20260811 22:26:01.523074 32363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42a677fee666457192ec1f4407ed69d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" is_pre_election: true
I20260811 22:26:01.523236 32363 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42a677fee666457192ec1f4407ed69d9 in term 2.
I20260811 22:26:01.523732 32533 leader_election.cc:302] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42a677fee666457192ec1f4407ed69d9, 8e796f4ce0654b669a4ed5f8ace642f8; no voters:
I20260811 22:26:01.523819 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.523583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55596)
0811 22:26:01.523699 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.523747 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:26:01.523962 32533 leader_election.cc:334] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027): Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
I20260811 22:26:01.523989 679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 22:26:01.524067 679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:26:01.524138 679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:26:01.525178 679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } }
I20260811 22:26:01.525508 679 leader_election.cc:288] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 3 election: Requested vote from peers 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027), 8e796f4ce0654b669a4ed5f8ace642f8 (127.24.71.253:35393)
I20260811 22:26:01.525884 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.525700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55608)
0811 22:26:01.525793 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.525836 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 22:26:01.526160 32363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42a677fee666457192ec1f4407ed69d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8e796f4ce0654b669a4ed5f8ace642f8"
I20260811 22:26:01.526309 32363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:26:01.526569 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.526361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55612)
0811 22:26:01.526473 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.526521 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:26:01.526769 32533 leader_election.cc:334] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027): Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
I20260811 22:26:01.527559 32363 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42a677fee666457192ec1f4407ed69d9 in term 3.
I20260811 22:26:01.527948 32533 leader_election.cc:302] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 42a677fee666457192ec1f4407ed69d9, 8e796f4ce0654b669a4ed5f8ace642f8; no voters: 9918892880914ddf8b203bc656c08730
I20260811 22:26:01.528172 679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:26:01.528460 679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [term 3 LEADER]: Becoming Leader. State: Replica: 42a677fee666457192ec1f4407ed69d9, State: Running, Role: LEADER
I20260811 22:26:01.528762 679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } }
I20260811 22:26:01.530462 682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42a677fee666457192ec1f4407ed69d9. Latest consensus state: current_term: 3 leader_uuid: "42a677fee666457192ec1f4407ed69d9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:26:01.530594 682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:01.531030 684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:01.533684 684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:01.534165 684 catalog_manager.cc:1259] Loaded cluster ID: c8e20ee46ee141e7bac9731a76061819
I20260811 22:26:01.534219 684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:01.534857 684 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:01.535316 684 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Loaded TSK: 0
I20260811 22:26:01.536134 684 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:26:01.612183 32549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:26:01.615561 32549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:26:01.619518 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.619207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55636)
0811 22:26:01.619354 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.619429 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 22:26:01.619680 32363 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [term 3 FOLLOWER]: Refusing update from remote peer 42a677fee666457192ec1f4407ed69d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 22:26:01.620503 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.620242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55650)
0811 22:26:01.620366 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.620426 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 22:26:01.620574 682 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 22:26:01.620788 32533 consensus_peers.cc:487] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 -> Peer 9918892880914ddf8b203bc656c08730 (127.24.71.254:35027): Couldn't send request to peer 9918892880914ddf8b203bc656c08730. Status: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:26:01.622232 686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42a677fee666457192ec1f4407ed69d9. Latest consensus state: current_term: 3 leader_uuid: "42a677fee666457192ec1f4407ed69d9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:26:01.622377 686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:01.623770 682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "42a677fee666457192ec1f4407ed69d9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:26:01.623952 682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:01.624095 686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "42a677fee666457192ec1f4407ed69d9" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9918892880914ddf8b203bc656c08730" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35027 } } peers { permanent_uuid: "8e796f4ce0654b669a4ed5f8ace642f8" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 35393 } attrs { promote: false } } peers { permanent_uuid: "42a677fee666457192ec1f4407ed69d9" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 34933 } attrs { promote: false } } }
I20260811 22:26:01.624233 686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e796f4ce0654b669a4ed5f8ace642f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:26:01.635499 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.631479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:41421 (local address 127.0.0.1:58846)
0811 22:26:01.631971 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.631982 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:01.632047 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:01.632453 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:01.632456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:01.632474 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:01.632588 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.632594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.633418 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.633424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.634256 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.634266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.635015 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.635019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.635124 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.635145 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:26:01.635239 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:26:01.635335 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":121,"threads_started":1}
I20260811 22:26:01.635499 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.631949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.24.71.193:41421)
0811 22:26:01.632226 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:01.632230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:01.632242 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:26:01.632290 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:01.632293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:01.632335 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:01.632436 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:01.632708 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.633297 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.634405 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.634876 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.635000 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.635075 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:01.635260 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 22:26:01.635347 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":82,"threads_started":1}
I20260811 22:26:01.636195 32677 tablet_service.cc:1467] Processing CreateTablet for tablet fba82f05ee4c40ec8093e3a123899c63 (DEFAULT_TABLE table=test-table [id=626b7c59b0a94965a958c6417c5e9a70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:01.636584 32677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba82f05ee4c40ec8093e3a123899c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:01.640107 698 tablet_bootstrap.cc:492] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278: Bootstrap starting.
I20260811 22:26:01.640923 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.631235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46249 (local address 127.0.0.1:37806)
0811 22:26:01.631630 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.631644 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:01.631755 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:01.637058 (+ 5303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:01.637063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:01.637083 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:01.637198 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.637204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.638354 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.638358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.639549 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.639560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.640506 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.640509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.640622 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.640651 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:26:01.640700 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 22:26:01.640777 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":58,"thread_start_us":111,"threads_started":1}
I20260811 22:26:01.641121 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.630809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:34493 (local address 127.0.0.1:59568)
0811 22:26:01.630987 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.631006 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:01.631116 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:01.636602 (+ 5486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:01.636609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:01.636631 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:01.636726 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.636733 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.638330 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.638334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.639608 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.639622 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.640649 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.640654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.640768 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.640800 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:26:01.640872 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:26:01.640957 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":1}
I20260811 22:26:01.641533 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.636195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37806 (local address 127.24.71.194:46249)
0811 22:26:01.636780 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:01.636785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:01.636800 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:01.636867 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:01.636872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:01.636928 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:01.637313 (+ 385us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:01.637418 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.638190 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.639695 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.640350 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.641203 (+ 853us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.641287 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:01.641324 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 22:26:01.641394 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":70,"threads_started":1}
I20260811 22:26:01.641700 695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.635970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59568 (local address 127.24.71.195:34493)
0811 22:26:01.636270 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:01.636275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:01.636292 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:26:01.636350 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:01.636353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:01.636422 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:01.637193 (+ 771us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:01.637317 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.638145 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.639793 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.640489 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.641345 (+ 856us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.641418 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:01.641461 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 22:26:01.641534 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":84,"threads_started":1}
I20260811 22:26:01.641839 698 tablet_bootstrap.cc:654] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:01.642127 360 tablet_service.cc:1467] Processing CreateTablet for tablet fba82f05ee4c40ec8093e3a123899c63 (DEFAULT_TABLE table=test-table [id=626b7c59b0a94965a958c6417c5e9a70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:01.642442 360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba82f05ee4c40ec8093e3a123899c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:01.642444 513 tablet_service.cc:1467] Processing CreateTablet for tablet fba82f05ee4c40ec8093e3a123899c63 (DEFAULT_TABLE table=test-table [id=626b7c59b0a94965a958c6417c5e9a70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:01.642752 513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba82f05ee4c40ec8093e3a123899c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:01.643342 698 tablet_bootstrap.cc:492] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278: No bootstrap required, opened a new log
I20260811 22:26:01.643427 698 ts_tablet_manager.cc:1397] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:01.643989 698 raft_consensus.cc:348] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } }
I20260811 22:26:01.644143 698 raft_consensus.cc:374] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:01.644199 698 raft_consensus.cc:729] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f107625eac4a55b7eb0301a96ff278, State: Initialized, Role: FOLLOWER
I20260811 22:26:01.644336 698 consensus_queue.cc:260] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } }
I20260811 22:26:01.644742 698 ts_tablet_manager.cc:1428] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:26:01.645697 701 tablet_bootstrap.cc:492] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3: Bootstrap starting.
I20260811 22:26:01.646654 701 tablet_bootstrap.cc:654] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:01.647300 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.646969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.193:41227)
0811 22:26:01.647153 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.647215 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 22:26:01.647475 702 tablet_bootstrap.cc:492] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: Bootstrap starting.
I20260811 22:26:01.647578 32752 heartbeater.cc:503] Master 127.24.71.252:34933 was elected leader, sending a full tablet report...
I20260811 22:26:01.648186 701 tablet_bootstrap.cc:492] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3: No bootstrap required, opened a new log
I20260811 22:26:01.648335 701 ts_tablet_manager.cc:1397] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:01.648793 701 raft_consensus.cc:348] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } }
I20260811 22:26:01.648960 701 raft_consensus.cc:374] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:01.649031 701 raft_consensus.cc:729] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbc9741b90d5474fa3268cca3bdbcfc3, State: Initialized, Role: FOLLOWER
I20260811 22:26:01.649181 701 consensus_queue.cc:260] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } }
I20260811 22:26:01.649596 701 ts_tablet_manager.cc:1428] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 22:26:01.650378 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.650019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.194:37673)
0811 22:26:01.650236 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.650299 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260811 22:26:01.651096 702 tablet_bootstrap.cc:654] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:01.651240 435 heartbeater.cc:503] Master 127.24.71.252:34933 was elected leader, sending a full tablet report...
I20260811 22:26:01.651921 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.651214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.194:50225)
0811 22:26:01.651764 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.651821 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479}
I20260811 22:26:01.653047 702 tablet_bootstrap.cc:492] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: No bootstrap required, opened a new log
I20260811 22:26:01.653205 702 ts_tablet_manager.cc:1397] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 22:26:01.653791 702 raft_consensus.cc:348] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } }
I20260811 22:26:01.653914 702 raft_consensus.cc:374] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:01.653961 702 raft_consensus.cc:729] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fceaadf920814f1ab7560a10d13a8385, State: Initialized, Role: FOLLOWER
I20260811 22:26:01.654111 702 consensus_queue.cc:260] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } }
I20260811 22:26:01.654637 702 ts_tablet_manager.cc:1428] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:26:01.655437 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.655154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.195:49233)
0811 22:26:01.655292 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.655363 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 22:26:01.656214 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.655985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.195:34203)
0811 22:26:01.656090 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.656149 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 22:26:01.656536 585 heartbeater.cc:503] Master 127.24.71.252:34933 was elected leader, sending a full tablet report...
W20260811 22:26:01.741353 32754 tablet.cc:2369] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:26:01.921285 706 raft_consensus.cc:482] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:26:01.921478 706 raft_consensus.cc:504] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } }
I20260811 22:26:01.922771 706 leader_election.cc:288] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbc9741b90d5474fa3268cca3bdbcfc3 (127.24.71.194:46249), 35f107625eac4a55b7eb0301a96ff278 (127.24.71.193:41421)
I20260811 22:26:01.926498 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.922993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46249 (local address 127.24.71.195:44703)
0811 22:26:01.923207 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.923223 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:01.923333 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:01.923635 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:01.923638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:01.923667 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:01.923798 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.923804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.924576 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.924582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.925382 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.925391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.926056 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.926060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.926163 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.926184 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:26:01.926269 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:26:01.926347 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":1,"spinlock_wait_cycles":896}
I20260811 22:26:01.926497 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.923136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:44703 (local address 127.24.71.194:46249)
0811 22:26:01.923304 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:01.923310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:01.923331 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:26:01.923480 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:01.923482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:01.923527 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:01.923617 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:01.923950 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.924466 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.925518 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.925915 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.926056 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.926114 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:01.926302 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 22:26:01.926382 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:26:01.926941 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.923239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:49471 (local address 127.24.71.193:41421)
0811 22:26:01.923374 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:01.923380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:01.923638 (+ 258us) server_negotiation.cc:407] Connection header received
0811 22:26:01.923790 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:01.923793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:01.923894 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:01.923989 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:01.924260 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.924823 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.925865 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.926284 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.926422 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.926474 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:01.926715 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 22:26:01.926821 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:26:01.927037 383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba82f05ee4c40ec8093e3a123899c63" candidate_uuid: "fceaadf920814f1ab7560a10d13a8385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" is_pre_election: true
I20260811 22:26:01.927222 383 raft_consensus.cc:2454] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fceaadf920814f1ab7560a10d13a8385 in term 0.
I20260811 22:26:01.927358 707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.923129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:41421 (local address 127.24.71.195:49471)
0811 22:26:01.923508 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.923519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:01.923656 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:01.924003 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:01.924006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:01.924021 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:01.924135 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.924141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.924949 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.924953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.925694 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:01.925703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:01.926418 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:01.926422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:01.926523 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:01.926572 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 22:26:01.927103 (+ 531us) client_negotiation.cc:240] Negotiation successful
0811 22:26:01.927215 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":163,"threads_started":1}
I20260811 22:26:01.927685 460 leader_election.cc:302] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbc9741b90d5474fa3268cca3bdbcfc3, fceaadf920814f1ab7560a10d13a8385; no voters:
I20260811 22:26:01.928013 706 raft_consensus.cc:2781] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:26:01.928092 706 raft_consensus.cc:482] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:26:01.928133 706 raft_consensus.cc:3037] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:01.928052 32698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba82f05ee4c40ec8093e3a123899c63" candidate_uuid: "fceaadf920814f1ab7560a10d13a8385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f107625eac4a55b7eb0301a96ff278" is_pre_election: true
I20260811 22:26:01.928297 32698 raft_consensus.cc:2454] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fceaadf920814f1ab7560a10d13a8385 in term 0.
I20260811 22:26:01.929454 706 raft_consensus.cc:504] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } }
I20260811 22:26:01.929818 706 leader_election.cc:288] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [CANDIDATE]: Term 1 election: Requested vote from peers dbc9741b90d5474fa3268cca3bdbcfc3 (127.24.71.194:46249), 35f107625eac4a55b7eb0301a96ff278 (127.24.71.193:41421)
I20260811 22:26:01.930188 32698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba82f05ee4c40ec8093e3a123899c63" candidate_uuid: "fceaadf920814f1ab7560a10d13a8385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f107625eac4a55b7eb0301a96ff278"
I20260811 22:26:01.930294 32698 raft_consensus.cc:3037] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:01.930222 383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba82f05ee4c40ec8093e3a123899c63" candidate_uuid: "fceaadf920814f1ab7560a10d13a8385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3"
I20260811 22:26:01.930493 383 raft_consensus.cc:3037] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:01.931355 32698 raft_consensus.cc:2454] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fceaadf920814f1ab7560a10d13a8385 in term 1.
I20260811 22:26:01.931516 383 raft_consensus.cc:2454] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fceaadf920814f1ab7560a10d13a8385 in term 1.
I20260811 22:26:01.931646 462 leader_election.cc:302] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f107625eac4a55b7eb0301a96ff278, fceaadf920814f1ab7560a10d13a8385; no voters:
I20260811 22:26:01.931871 706 raft_consensus.cc:2781] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:01.932119 706 raft_consensus.cc:686] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 1 LEADER]: Becoming Leader. State: Replica: fceaadf920814f1ab7560a10d13a8385, State: Running, Role: LEADER
I20260811 22:26:01.932411 706 consensus_queue.cc:237] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } }
I20260811 22:26:01.932893 707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.932641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.195:58661)
0811 22:26:01.932749 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.932819 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 22:26:01.935184 32549 catalog_manager.cc:5495] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 reported cstate change: term changed from 0 to 1, leader changed from <none> to fceaadf920814f1ab7560a10d13a8385 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "fceaadf920814f1ab7560a10d13a8385" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceaadf920814f1ab7560a10d13a8385" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 } health_report { overall_health: UNKNOWN } } }
I20260811 22:26:01.998739 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.998409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55656)
0811 22:26:01.998570 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.998643 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 22:26:01.999588 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:01.999352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.0.0.1:55662)
0811 22:26:01.999471 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:01.999525 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 22:26:02.015693 695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:02.011195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.24.71.195:34493)
0811 22:26:02.011404 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:02.011408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:02.011423 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:02.011515 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:02.011517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:02.011585 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:02.011694 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:02.012166 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:02.012730 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:02.013704 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:02.014066 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:02.014175 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:02.014545 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:02.015139 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:02.015237 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:02.015365 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 22:26:02.015505 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:26:02.019333 383 raft_consensus.cc:1261] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 1 FOLLOWER]: Refusing update from remote peer fceaadf920814f1ab7560a10d13a8385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:02.019987 32698 raft_consensus.cc:1261] T fba82f05ee4c40ec8093e3a123899c63 P 35f107625eac4a55b7eb0301a96ff278 [term 1 FOLLOWER]: Refusing update from remote peer fceaadf920814f1ab7560a10d13a8385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:02.020174 708 consensus_queue.cc:1035] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbc9741b90d5474fa3268cca3bdbcfc3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:02.020653 706 consensus_queue.cc:1035] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f107625eac4a55b7eb0301a96ff278" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:02.022064 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:02.021726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.194:56981)
0811 22:26:02.021896 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:02.021977 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 22:26:02.022733 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:02.022400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.193:47273)
0811 22:26:02.022597 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:02.022656 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 22:26:02.027232 707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:02.026832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.195:49265)
0811 22:26:02.027078 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:02.027147 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260811 22:26:02.028612 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:02.028208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.193:36823)
0811 22:26:02.028463 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:02.028526 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260811 22:26:02.028970 717 mvcc.cc:205] Tried to move back new op lower bound from 7317451415624835072 to 7317451415278108672. Current Snapshot: MvccSnapshot[applied={T|T < 7317451415624835072 or (T in {7317451415624835072})}]
I20260811 22:26:02.029443 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:02.028976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:35027 (local address 127.24.71.194:44007)
0811 22:26:02.029301 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:02.029357 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:35027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252}
I20260811 22:26:02.030994 713 mvcc.cc:205] Tried to move back new op lower bound from 7317451415624835072 to 7317451415278108672. Current Snapshot: MvccSnapshot[applied={T|T < 7317451415624835072}]
I20260811 22:26:02.036372 32549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:02.036580 32549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:02.040954 32549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42a677fee666457192ec1f4407ed69d9: Sending DeleteTablet for 3 replicas of tablet fba82f05ee4c40ec8093e3a123899c63
I20260811 22:26:02.042137 32677 tablet_service.cc:1514] Processing DeleteTablet for tablet fba82f05ee4c40ec8093e3a123899c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:02 UTC) from {username='oryx', principal='oryx/127.24.71.252@KRBTEST.COM'} at 127.0.0.1:58846
I20260811 22:26:02.042369 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 32508
I20260811 22:26:02.042570 360 tablet_service.cc:1514] Processing DeleteTablet for tablet fba82f05ee4c40ec8093e3a123899c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:02 UTC) from {username='oryx', principal='oryx/127.24.71.252@KRBTEST.COM'} at 127.0.0.1:37806
I20260811 22:26:02.046367 513 tablet_service.cc:1514] Processing DeleteTablet for tablet fba82f05ee4c40ec8093e3a123899c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:02 UTC) from {username='oryx', principal='oryx/127.24.71.252@KRBTEST.COM'} at 127.0.0.1:59568
I20260811 22:26:02.046851 728 tablet_replica.cc:330] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3: stopping tablet replica
I20260811 22:26:02.047034 728 raft_consensus.cc:2229] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:02.047230 728 raft_consensus.cc:2258] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:02.052646 729 tablet_replica.cc:330] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: stopping tablet replica
I20260811 22:26:02.052887 729 raft_consensus.cc:2229] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:26:02.053424 729 raft_consensus.cc:2258] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:02.053784 728 ts_tablet_manager.cc:1905] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:02.054297 729 ts_tablet_manager.cc:1905] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:26:02.057058 680 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:02.056325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:41421 (local address 127.0.0.1:58852)
0811 22:26:02.056677 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:02.056692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:02.056796 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:02.056910 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:41421: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":268}
I20260811 22:26:02.057178 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 32759
I20260811 22:26:02.057299 728 ts_tablet_manager.cc:1918] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 22:26:02.057293 32535 catalog_manager.cc:4584] TS 35f107625eac4a55b7eb0301a96ff278 (127.24.71.193:41421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fba82f05ee4c40ec8093e3a123899c63: Network error: Client connection negotiation failed: client connection to 127.24.71.193:41421: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260811 22:26:02.057389 728 log.cc:1196] T fba82f05ee4c40ec8093e3a123899c63 P dbc9741b90d5474fa3268cca3bdbcfc3: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/fba82f05ee4c40ec8093e3a123899c63
I20260811 22:26:02.058640 729 ts_tablet_manager.cc:1918] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:26:02.058800 729 log.cc:1196] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/fba82f05ee4c40ec8093e3a123899c63
I20260811 22:26:02.059151 729 ts_tablet_manager.cc:1939] T fba82f05ee4c40ec8093e3a123899c63 P fceaadf920814f1ab7560a10d13a8385: Deleting consensus metadata
I20260811 22:26:02.060185 32535 catalog_manager.cc:4841] TS fceaadf920814f1ab7560a10d13a8385 (127.24.71.195:34493): tablet fba82f05ee4c40ec8093e3a123899c63 (table test-table [id=626b7c59b0a94965a958c6417c5e9a70]) successfully deleted
I20260811 22:26:02.065923 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 442
I20260811 22:26:02.065999 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:02.065684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46249 (local address 127.0.0.1:37818)
0811 22:26:02.065834 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:02.065900 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.194:46249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:26:02.075251 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 32202
I20260811 22:26:02.085202 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 32311
2026-08-11T22:26:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7622 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:26:02 dist-test-slave-mvvj krb5kdc[732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:26:02 dist-test-slave-mvvj krb5kdc[732](info): set up 2 sockets
Aug 11 22:26:02 dist-test-slave-mvvj krb5kdc[732](info): commencing operation
krb5kdc: starting...
W20260811 22:26:03.235991 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:26:03 dist-test-slave-mvvj krb5kdc[732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:26:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.254.keytab.
Entry for principal oryx/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.254.keytab.
I20260811 22:26:03.380192 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:36661
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.254.keytab
--principal=oryx/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:36661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:03.501660 746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:03.502012 746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:03.502135 746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:03.506577 746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:03.506685 746 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:26:03.506718 746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:03.506762 746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:03.506794 746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:03.506824 746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:03.512706 746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:36661
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.254.keytab
--principal=oryx/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:36661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:03.513928 746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:03.515055 746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:03.524895 753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:03.524914 752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:03.524895 756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:03.525278 746 server_base.cc:1034] running on GCE node
I20260811 22:26:03.525991 746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:03.527369 746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:03.528589 746 hybrid_clock.cc:648] HybridClock initialized: now 1786487163528574 us; error 31 us; skew 500 ppm
Aug 11 22:26:03 dist-test-slave-mvvj krb5kdc[732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:03.537972 746 init.cc:377] Logged in from keytab as oryx/127.24.71.254@KRBTEST.COM (short username oryx)
I20260811 22:26:03.539192 746 webserver.cc:460] Webserver started at http://127.24.71.254:43597/ using document root <none> and password file <none>
I20260811 22:26:03.539911 746 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:03.539987 746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:03.540230 746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:03.542531 746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-mvvj"
server_key: "f3878ce85bfbdd9fa6146c8ee04286a7"
server_key_iv: "0bb5e7ba11d9cc4a2ccc7a101216c131"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.543220 746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-mvvj"
server_key: "f3878ce85bfbdd9fa6146c8ee04286a7"
server_key_iv: "0bb5e7ba11d9cc4a2ccc7a101216c131"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.547933 746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:26:03.551035 763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:03.552942 746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:26:03.553155 746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-mvvj"
server_key: "f3878ce85bfbdd9fa6146c8ee04286a7"
server_key_iv: "0bb5e7ba11d9cc4a2ccc7a101216c131"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.553303 746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:03.579160 746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:03.582793 746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:03.583045 746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:03.592514 746 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:36661
I20260811 22:26:03.592532 829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:36661 every 8 connection(s)
I20260811 22:26:03.593645 746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:26:03.596988 831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:03.598770 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 746
I20260811 22:26:03.598943 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:26:03.599261 24863 external_mini_cluster.cc:1428] Setting key d9ada6c271d1f7b58c3e46a4ca68ac8d
I20260811 22:26:03.604097 831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564: Bootstrap starting.
I20260811 22:26:03.607331 831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:03.608788 831 log.cc:824] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:26:03 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:26:03.611634 831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564: No bootstrap required, opened a new log
I20260811 22:26:03.616700 831 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36661 } }
I20260811 22:26:03.617077 831 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:03.617151 831 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7fd5b7a87874f8c9a94fae1a1dd3564, State: Initialized, Role: FOLLOWER
I20260811 22:26:03.617690 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.600864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56028 (local address 127.24.71.254:36661)
0811 22:26:03.601325 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.601342 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.601381 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:26:03.602053 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.602071 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.602418 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.602699 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.603677 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.604897 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.606168 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.607036 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.607235 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.610922 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.610930 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.610951 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.610989 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.613361 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.614061 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.614068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.614076 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.614256 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.614655 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.614658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.614660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.615280 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.615520 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.615979 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.616349 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":149,"threads_started":1}
I20260811 22:26:03.617697 831 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36661 } }
I20260811 22:26:03.617903 831 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:26:03.617990 831 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:26:03.618103 831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:03.619236 831 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36661 } }
I20260811 22:26:03.619639 831 leader_election.cc:302] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7fd5b7a87874f8c9a94fae1a1dd3564; no voters:
I20260811 22:26:03.620038 831 leader_election.cc:288] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:26:03.620165 837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:03.620432 837 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [term 1 LEADER]: Becoming Leader. State: Replica: e7fd5b7a87874f8c9a94fae1a1dd3564, State: Running, Role: LEADER
I20260811 22:26:03.620824 837 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36661 } }
I20260811 22:26:03.621521 831 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:03.622277 838 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7fd5b7a87874f8c9a94fae1a1dd3564. Latest consensus state: current_term: 1 leader_uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36661 } } }
I20260811 22:26:03.622567 838 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:03.622759 839 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fd5b7a87874f8c9a94fae1a1dd3564" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36661 } } }
I20260811 22:26:03.622875 839 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564 [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:03.623507 846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:03.627902 846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:03.637625 846 catalog_manager.cc:1347] Generated new cluster ID: 5ccefd75e3274463ac43f825e2e91c76
I20260811 22:26:03.637748 846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:03.676232 846 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:26:03.677311 846 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:03.684178 846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564: Generated new TSK 0
I20260811 22:26:03.685006 846 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab.
Entry for principal oryx/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab.
I20260811 22:26:03.724227 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:36661
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab
--principal=oryx/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:03.843040 858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:03.843326 858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:03.843422 858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:03.847988 858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:03.848083 858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:03.848127 858 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:26:03.848213 858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:26:03.854162 858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab
--principal=oryx/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.71.254:36661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:03.855475 858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:03.856667 858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:03.868139 863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:03.868180 864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:03.868458 858 server_base.cc:1034] running on GCE node
W20260811 22:26:03.868180 866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:03.868927 858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:03.869655 858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:03.870884 858 hybrid_clock.cc:648] HybridClock initialized: now 1786487163870863 us; error 28 us; skew 500 ppm
Aug 11 22:26:03 dist-test-slave-mvvj krb5kdc[732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:03.874150 858 init.cc:377] Logged in from keytab as oryx/127.24.71.193@KRBTEST.COM (short username oryx)
I20260811 22:26:03.875228 858 webserver.cc:460] Webserver started at http://127.24.71.193:38001/ using document root <none> and password file <none>
I20260811 22:26:03.875880 858 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:03.875980 858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:03.876228 858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:03.878407 858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "82e38521232d4cccb66debda5a10f7bb"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-mvvj"
server_key: "30c71b58e712e9a03f05b77c8814f367"
server_key_iv: "73f10b494a080258069a0b5558d60ea6"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.879024 858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "82e38521232d4cccb66debda5a10f7bb"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-mvvj"
server_key: "30c71b58e712e9a03f05b77c8814f367"
server_key_iv: "73f10b494a080258069a0b5558d60ea6"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.883819 858 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 22:26:03.886826 873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:03.888509 858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:03.888681 858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "82e38521232d4cccb66debda5a10f7bb"
format_stamp: "Formatted at 2026-08-11 22:26:03 on dist-test-slave-mvvj"
server_key: "30c71b58e712e9a03f05b77c8814f367"
server_key_iv: "73f10b494a080258069a0b5558d60ea6"
server_key_version: "encryptionkey@0"
I20260811 22:26:03.888825 858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:03.924723 858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:03.928122 858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:03.928369 858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:03.929960 858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:03.930043 858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:03.930125 858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:03.930171 858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:26:03 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:26:03.942312 858 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:40637
I20260811 22:26:03.942318 1001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:40637 every 8 connection(s)
I20260811 22:26:03.943318 858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:26:03.945566 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.931010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:35357 (local address 127.24.71.254:36661)
0811 22:26:03.931147 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.931151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.932373 (+ 1222us) server_negotiation.cc:407] Connection header received
0811 22:26:03.933671 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.933676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.933752 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.933874 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.935386 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.936010 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.937272 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.937655 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.937788 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.941190 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.941194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.941197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.941235 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.942962 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.943530 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.943534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.943536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.943646 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.944201 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.944205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.944206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.944481 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.944576 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.945248 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.945437 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:26:03.946274 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.931617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36661 (local address 127.24.71.193:35357)
0811 22:26:03.932193 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.932240 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.933369 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.934083 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.934093 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.934535 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.935202 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.935223 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.936155 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.936158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.937112 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.937122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.937856 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.937861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.938060 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.938772 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.938799 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.940990 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.943100 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.943106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.943123 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.943402 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.943800 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.943864 (+ 64us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.943866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.944056 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.944621 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.944627 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.944900 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.945068 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.945342 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":187,"threads_started":1}
I20260811 22:26:03.947633 1002 heartbeater.cc:347] Connected to a master server at 127.24.71.254:36661
I20260811 22:26:03.947789 1002 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:03.948447 1002 heartbeater.cc:511] Master 127.24.71.254:36661 requested a full tablet report, sending...
I20260811 22:26:03.950516 780 ts_manager.cc:194] Registered new tserver with Master: 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637)
I20260811 22:26:03.951944 780 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:35357
I20260811 22:26:03.953017 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 858
I20260811 22:26:03.953183 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:26:03.953545 24863 external_mini_cluster.cc:1428] Setting key 1aed3172cd38c38a152f9d56a23ed94d
I20260811 22:26:03.959573 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.950234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36661 (local address 127.24.71.193:54691)
0811 22:26:03.950718 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:03.950734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:03.950866 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:03.951167 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:03.951170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:03.951400 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:03.951516 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.951522 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.952427 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.952431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.953213 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:03.953225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.953880 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.953884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:03.953975 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.954967 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:03.954980 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:03.955326 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:03.957499 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.957506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.957511 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.957874 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.958290 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:03.958310 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:03.958315 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:03.958483 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:03.959012 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:03.959015 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:03.959109 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 22:26:03.959200 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:26:03.959333 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":111,"threads_started":1}
I20260811 22:26:03.960017 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:03.950480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:54691 (local address 127.24.71.254:36661)
0811 22:26:03.950623 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:03.950627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:03.950815 (+ 188us) server_negotiation.cc:407] Connection header received
0811 22:26:03.951008 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:03.951010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:03.951054 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:03.951146 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:03.951646 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.952287 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.953408 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:03.953726 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:03.953841 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:03.955470 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:03.955474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:03.955477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:03.955505 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:03.957375 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.957982 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.957989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.957991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.958103 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:03.958597 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:03.958604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:03.958607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:03.958887 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:03.959012 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:03.959305 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 22:26:03.959453 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for oryx/127.24.71.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab.
Entry for principal oryx/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab.
I20260811 22:26:03.991106 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:36661
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab
--principal=oryx/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:04.118757 1012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:04.119092 1012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:04.119179 1012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:04.123893 1012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:04.123978 1012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:04.124055 1012 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:26:04.124130 1012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:26:04.129850 1012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab
--principal=oryx/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.71.254:36661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:04.131549 1012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:04.133330 1012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:04.143816 1020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:04.143996 1018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:04.143818 1023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:04.144644 1012 server_base.cc:1034] running on GCE node
I20260811 22:26:04.145112 1012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:04.145807 1012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:04.147029 1012 hybrid_clock.cc:648] HybridClock initialized: now 1786487164146996 us; error 50 us; skew 500 ppm
Aug 11 22:26:04 dist-test-slave-mvvj krb5kdc[732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487164, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:04.150949 1012 init.cc:377] Logged in from keytab as oryx/127.24.71.194@KRBTEST.COM (short username oryx)
I20260811 22:26:04.152091 1012 webserver.cc:460] Webserver started at http://127.24.71.194:40835/ using document root <none> and password file <none>
I20260811 22:26:04.152724 1012 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:04.152835 1012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:04.153103 1012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:04.155391 1012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "08991788adaa4e9cabe6706a70e3b6f3"
format_stamp: "Formatted at 2026-08-11 22:26:04 on dist-test-slave-mvvj"
server_key: "15b80af64226cf051a9474c9e8db61c9"
server_key_iv: "d4babe4784bf615bfa0175a521a8b214"
server_key_version: "encryptionkey@0"
I20260811 22:26:04.156152 1012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "08991788adaa4e9cabe6706a70e3b6f3"
format_stamp: "Formatted at 2026-08-11 22:26:04 on dist-test-slave-mvvj"
server_key: "15b80af64226cf051a9474c9e8db61c9"
server_key_iv: "d4babe4784bf615bfa0175a521a8b214"
server_key_version: "encryptionkey@0"
I20260811 22:26:04.165096 1012 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260811 22:26:04.168572 1030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:04.170288 1012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:26:04.170460 1012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "08991788adaa4e9cabe6706a70e3b6f3"
format_stamp: "Formatted at 2026-08-11 22:26:04 on dist-test-slave-mvvj"
server_key: "15b80af64226cf051a9474c9e8db61c9"
server_key_iv: "d4babe4784bf615bfa0175a521a8b214"
server_key_version: "encryptionkey@0"
I20260811 22:26:04.170605 1012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:04.186695 1012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:04.190376 1012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:04.190596 1012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:04.192252 1012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:04.192332 1012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:04.192413 1012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:04.192468 1012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:04.204428 1012 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:44717
I20260811 22:26:04.205470 1012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:26:04.205492 1159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:44717 every 8 connection(s)
Aug 11 22:26:04 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487164, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:26:04.210273 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 1012
I20260811 22:26:04.210497 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:26:04.210925 24863 external_mini_cluster.cc:1428] Setting key 3f9220dc680ce52f30be5ee3c2f14be3
I20260811 22:26:04.211763 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.193537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:34883 (local address 127.24.71.254:36661)
0811 22:26:04.193712 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.193719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.195044 (+ 1325us) server_negotiation.cc:407] Connection header received
0811 22:26:04.196705 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.196710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.196793 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.196961 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.199050 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.200072 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.202208 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.202656 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.202787 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.206664 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:04.206673 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:04.206678 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:04.206720 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:04.208828 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.209412 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.209418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.209421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.209544 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.210020 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.210031 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.210034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.210327 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:04.210450 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.211340 (+ 890us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.211516 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:26:04.212919 1044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.194154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36661 (local address 127.24.71.194:34883)
0811 22:26:04.194811 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.194883 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.196349 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.197308 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.197321 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.197936 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:04.198790 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.198816 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.200240 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.200245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.201991 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.202005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.203076 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.203082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.203351 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.204301 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:04.204330 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:04.206447 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:04.208980 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.208986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.208999 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.209279 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.209683 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.209686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.209687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.209875 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.210537 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:04.210545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:04.211007 (+ 462us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.211265 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.211677 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":141,"threads_started":1}
I20260811 22:26:04.215358 1160 heartbeater.cc:347] Connected to a master server at 127.24.71.254:36661
I20260811 22:26:04.215552 1160 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:04.216339 1160 heartbeater.cc:511] Master 127.24.71.254:36661 requested a full tablet report, sending...
I20260811 22:26:04.217656 780 ts_manager.cc:194] Registered new tserver with Master: 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717)
I20260811 22:26:04.218359 780 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:34883
WARNING: no policy specified for oryx/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:26:04.227448 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.217348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:49235 (local address 127.24.71.254:36661)
0811 22:26:04.218746 (+ 1398us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.218751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.218767 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:26:04.218812 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.218815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.218860 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.218944 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.219699 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.220557 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.221656 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.221960 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.222080 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.223509 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:04.223513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:04.223516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:04.223545 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:04.225355 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.225850 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.225854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.225856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.225951 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.226425 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.226428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.226429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.226681 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:04.226812 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.227114 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.227242 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321}
I20260811 22:26:04.227455 1167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.217311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36661 (local address 127.24.71.194:49235)
0811 22:26:04.217726 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.217743 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.217854 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.219143 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.219146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.219412 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:04.219539 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.219548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.220742 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.220745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.221510 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.221522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.222101 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.222105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.222206 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.222954 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:04.222968 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:04.223347 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:04.225480 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.225484 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.225487 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.225716 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.226054 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.226058 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.226060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.226158 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.226807 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:04.226811 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:04.226924 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.227048 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.227229 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab.
Entry for principal oryx/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab.
I20260811 22:26:04.251996 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:36661
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab
--principal=oryx/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:04.372997 1169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:04.373296 1169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:04.373347 1169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:04.377990 1169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:04.378074 1169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:04.378119 1169 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:26:04.378157 1169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:26:04.384018 1169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab
--principal=oryx/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.71.254:36661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:04.385160 1169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:04.386216 1169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:04.396607 1179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:04.396664 1177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:04.396607 1176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:04.396919 1169 server_base.cc:1034] running on GCE node
I20260811 22:26:04.397454 1169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:04.398175 1169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:04.399410 1169 hybrid_clock.cc:648] HybridClock initialized: now 1786487164399392 us; error 30 us; skew 500 ppm
Aug 11 22:26:04 dist-test-slave-mvvj krb5kdc[732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487164, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:04.402495 1169 init.cc:377] Logged in from keytab as oryx/127.24.71.195@KRBTEST.COM (short username oryx)
I20260811 22:26:04.403466 1169 webserver.cc:460] Webserver started at http://127.24.71.195:36371/ using document root <none> and password file <none>
I20260811 22:26:04.404105 1169 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:04.404182 1169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:04.404378 1169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:04.406540 1169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "c5e5d5a5395349f6adc7ed239db856c7"
format_stamp: "Formatted at 2026-08-11 22:26:04 on dist-test-slave-mvvj"
server_key: "c49007cbdfd3aa9a8a0f1117e21089ad"
server_key_iv: "8274585f3b269f3ae9fe3ba1e1a1dca0"
server_key_version: "encryptionkey@0"
I20260811 22:26:04.407078 1169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "c5e5d5a5395349f6adc7ed239db856c7"
format_stamp: "Formatted at 2026-08-11 22:26:04 on dist-test-slave-mvvj"
server_key: "c49007cbdfd3aa9a8a0f1117e21089ad"
server_key_iv: "8274585f3b269f3ae9fe3ba1e1a1dca0"
server_key_version: "encryptionkey@0"
I20260811 22:26:04.411101 1169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:26:04.413913 1186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:04.415407 1169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:04.415581 1169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "c5e5d5a5395349f6adc7ed239db856c7"
format_stamp: "Formatted at 2026-08-11 22:26:04 on dist-test-slave-mvvj"
server_key: "c49007cbdfd3aa9a8a0f1117e21089ad"
server_key_iv: "8274585f3b269f3ae9fe3ba1e1a1dca0"
server_key_version: "encryptionkey@0"
I20260811 22:26:04.415702 1169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:04.433367 1169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:04.436650 1169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:04.436874 1169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:04.438444 1169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:04.438525 1169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:04.438598 1169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:04.438653 1169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:26:04 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487164, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:26:04.450462 1169 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:37369
I20260811 22:26:04.450519 1314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:37369 every 8 connection(s)
I20260811 22:26:04.451680 1169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:26:04.453830 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.439373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:43611 (local address 127.24.71.254:36661)
0811 22:26:04.439572 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.439580 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.440505 (+ 925us) server_negotiation.cc:407] Connection header received
0811 22:26:04.441624 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.441630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.441719 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.441786 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.443484 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.444330 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.445546 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.445903 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.446049 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.449581 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:04.449585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:04.449587 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:04.449625 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:04.451467 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.452130 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.452134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.452136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.452236 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.452641 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.452644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.452646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.452854 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:04.452965 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.453481 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.453659 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:26:04.454566 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.439810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36661 (local address 127.24.71.195:43611)
0811 22:26:04.440332 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.440383 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.441363 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.442278 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.442289 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.442715 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:04.443313 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.443339 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.444459 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.444465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.445402 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.445413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.446111 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.446119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.446380 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.447128 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:04.447148 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:04.449378 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:04.451635 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.451640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.451653 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.452018 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.452350 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.452353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.452354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.452519 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.452975 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:04.452984 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:04.453236 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.453420 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.453750 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":138,"threads_started":1}
I20260811 22:26:04.455910 1315 heartbeater.cc:347] Connected to a master server at 127.24.71.254:36661
I20260811 22:26:04.456128 1315 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:04.456959 1315 heartbeater.cc:511] Master 127.24.71.254:36661 requested a full tablet report, sending...
I20260811 22:26:04.458187 781 ts_manager.cc:194] Registered new tserver with Master: c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369)
I20260811 22:26:04.458953 781 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:43611
I20260811 22:26:04.460173 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 1169
I20260811 22:26:04.460314 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:26:04.460693 24863 external_mini_cluster.cc:1428] Setting key eeba2de1f5f980b0a0253b3dc83aa387
I20260811 22:26:04.462996 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:26:04.466503 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.458374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:48159 (local address 127.24.71.254:36661)
0811 22:26:04.458495 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.458499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.458876 (+ 377us) server_negotiation.cc:407] Connection header received
0811 22:26:04.458932 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.458935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.458989 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.459082 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.459628 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.460310 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.461337 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.461618 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.461738 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.463095 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:04.463099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:04.463102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:04.463139 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:04.464719 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.465171 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.465175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.465177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.465263 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.465567 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.465572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.465577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.465837 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:04.465956 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.466225 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.466339 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:26:04.466521 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.458480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36661 (local address 127.24.71.195:48159)
0811 22:26:04.458765 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.458781 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.458873 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.459114 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.459118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.459356 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:04.459470 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.459476 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.460467 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.460470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.461202 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.461216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.461764 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.461768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.461858 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.462501 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:04.462513 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:04.462947 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:04.464844 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.464848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.464850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.465062 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.465371 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.465374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.465376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.465467 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.465947 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:04.465950 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:04.466053 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.466166 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.466319 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":7,"thread_start_us":126,"threads_started":1}
Aug 11 22:26:04 dist-test-slave-mvvj krb5kdc[732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487164, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:26:04 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:04 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:26:04.489791 835 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:04.483532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.24.71.254:36661)
0811 22:26:04.483672 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.483675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.483689 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:04.483733 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.483736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.483776 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.483923 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.484638 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.485182 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.486043 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.486256 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.486339 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.489669 (+ 3330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56036: BlockingRecv error: recv got EOF from 127.0.0.1:56036 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 22:26:04 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487164, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:26:04.499462 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.491332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.24.71.254:36661)
0811 22:26:04.491476 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.491479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.491596 (+ 117us) server_negotiation.cc:407] Connection header received
0811 22:26:04.491696 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.491698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.491738 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.491883 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.492466 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.493036 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.493895 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.494114 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.494197 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.496466 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:04.496469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:04.496471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:04.496496 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:04.497851 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.498246 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.498248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.498250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.498399 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.498668 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.498671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.498672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.498864 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:04.498972 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.499205 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.499305 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:26:04.502575 781 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:26:04.505787 781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:26:04.520980 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.515001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39642 (local address 127.24.71.193:40637)
0811 22:26:04.515704 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.515711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.515736 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:26:04.515890 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.515896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.516111 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.516709 (+ 598us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:04.516884 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.517710 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.519395 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.520259 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.520378 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.520548 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.520732 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.520838 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":83,"threads_started":1}
I20260811 22:26:04.521219 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.515538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:44717 (local address 127.0.0.1:60152)
0811 22:26:04.516151 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.516164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.516235 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.516906 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.516909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.516938 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:04.517037 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.517044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.518381 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.518384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.519632 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.519643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.520755 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.520758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.520888 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.520910 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.520976 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.521054 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":78,"thread_start_us":98,"threads_started":1}
I20260811 22:26:04.521791 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.515000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:37369 (local address 127.0.0.1:58180)
0811 22:26:04.515861 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.515888 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.515986 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.516587 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.516590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.516609 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:04.516721 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.516728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.518310 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.518314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.519612 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.519622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.520754 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.520766 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.521433 (+ 667us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.521466 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.521569 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.521661 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":602,"thread_start_us":98,"threads_started":1}
I20260811 22:26:04.521834 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.515623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58180 (local address 127.24.71.195:37369)
0811 22:26:04.516107 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.516119 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.516140 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:26:04.516217 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.516222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.516420 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.516879 (+ 459us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:04.517011 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.518167 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.519720 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.520607 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.520866 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.521046 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.521607 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.521700 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":132,"threads_started":1}
I20260811 22:26:04.522055 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.514817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40637 (local address 127.0.0.1:39642)
0811 22:26:04.515347 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.515399 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.515531 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.516301 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.516307 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.516333 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:04.516530 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.516540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.517829 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.517834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.519250 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.519264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.520392 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.520395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.520516 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.520557 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.520699 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.521954 (+ 1255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":1133,"thread_start_us":90,"threads_started":1}
I20260811 22:26:04.522854 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.516096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.24.71.194:44717)
0811 22:26:04.516482 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.516495 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.516520 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:26:04.516591 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.516596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.516766 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.516905 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:04.517163 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.518252 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.519774 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.520610 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.522345 (+ 1735us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.522581 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.522631 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.522721 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":117,"threads_started":1}
I20260811 22:26:04.525277 1245 tablet_service.cc:1467] Processing CreateTablet for tablet 21ac4d84e93c4d09aa2a196a24bed646 (DEFAULT_TABLE table=test-table [id=1b39e87a3a2847ddbe248b57d72fe7a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:04.525264 927 tablet_service.cc:1467] Processing CreateTablet for tablet 21ac4d84e93c4d09aa2a196a24bed646 (DEFAULT_TABLE table=test-table [id=1b39e87a3a2847ddbe248b57d72fe7a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:04.525991 1087 tablet_service.cc:1467] Processing CreateTablet for tablet 21ac4d84e93c4d09aa2a196a24bed646 (DEFAULT_TABLE table=test-table [id=1b39e87a3a2847ddbe248b57d72fe7a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:04.526463 1245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ac4d84e93c4d09aa2a196a24bed646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:04.526463 927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ac4d84e93c4d09aa2a196a24bed646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:04.526968 1087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ac4d84e93c4d09aa2a196a24bed646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:04.534490 1347 tablet_bootstrap.cc:492] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: Bootstrap starting.
I20260811 22:26:04.534845 1349 tablet_bootstrap.cc:492] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: Bootstrap starting.
I20260811 22:26:04.536190 1350 tablet_bootstrap.cc:492] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: Bootstrap starting.
I20260811 22:26:04.536614 1347 tablet_bootstrap.cc:654] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:04.536755 1349 tablet_bootstrap.cc:654] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:04.537418 1347 log.cc:824] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:04.537616 1349 log.cc:824] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:04.538290 1350 tablet_bootstrap.cc:654] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:04.539098 1350 log.cc:824] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:04.540145 1349 tablet_bootstrap.cc:492] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: No bootstrap required, opened a new log
I20260811 22:26:04.540145 1347 tablet_bootstrap.cc:492] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: No bootstrap required, opened a new log
I20260811 22:26:04.540354 1347 ts_tablet_manager.cc:1397] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:26:04.540354 1349 ts_tablet_manager.cc:1397] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:26:04.540928 1350 tablet_bootstrap.cc:492] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: No bootstrap required, opened a new log
I20260811 22:26:04.541141 1350 ts_tablet_manager.cc:1397] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:26:04.544024 1350 raft_consensus.cc:348] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } }
I20260811 22:26:04.544281 1350 raft_consensus.cc:374] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:04.544334 1350 raft_consensus.cc:729] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e5d5a5395349f6adc7ed239db856c7, State: Initialized, Role: FOLLOWER
I20260811 22:26:04.544564 1349 raft_consensus.cc:348] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } }
I20260811 22:26:04.544564 1347 raft_consensus.cc:348] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } }
I20260811 22:26:04.544814 1347 raft_consensus.cc:374] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:04.544816 1349 raft_consensus.cc:374] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:04.544867 1347 raft_consensus.cc:729] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08991788adaa4e9cabe6706a70e3b6f3, State: Initialized, Role: FOLLOWER
I20260811 22:26:04.544885 1349 raft_consensus.cc:729] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e38521232d4cccb66debda5a10f7bb, State: Initialized, Role: FOLLOWER
I20260811 22:26:04.544843 1350 consensus_queue.cc:260] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } }
I20260811 22:26:04.545305 1347 consensus_queue.cc:260] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } }
I20260811 22:26:04.545305 1349 consensus_queue.cc:260] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } }
I20260811 22:26:04.546059 1350 ts_tablet_manager.cc:1428] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:26:04.546236 1349 ts_tablet_manager.cc:1428] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:26:04.546236 1347 ts_tablet_manager.cc:1428] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:26:04.546391 1315 heartbeater.cc:503] Master 127.24.71.254:36661 was elected leader, sending a full tablet report...
I20260811 22:26:04.546567 1002 heartbeater.cc:503] Master 127.24.71.254:36661 was elected leader, sending a full tablet report...
I20260811 22:26:04.546579 1160 heartbeater.cc:503] Master 127.24.71.254:36661 was elected leader, sending a full tablet report...
I20260811 22:26:04.555948 1355 raft_consensus.cc:482] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:26:04.556149 1355 raft_consensus.cc:504] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } }
I20260811 22:26:04.557339 1355 leader_election.cc:288] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369), 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717)
I20260811 22:26:04.561462 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.557621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:51501 (local address 127.24.71.195:37369)
0811 22:26:04.557763 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.557769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.558052 (+ 283us) server_negotiation.cc:407] Connection header received
0811 22:26:04.558189 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.558192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.558249 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.558342 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:04.558671 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.559190 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.560468 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.560878 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.561012 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.561066 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.561255 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.561327 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:26:04.561522 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.557855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:40163 (local address 127.24.71.194:44717)
0811 22:26:04.558021 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.558028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.558155 (+ 127us) server_negotiation.cc:407] Connection header received
0811 22:26:04.558262 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.558264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.558343 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.558412 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:04.558821 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.559349 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.560533 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.560951 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.561083 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.561137 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.561325 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.561395 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:26:04.561568 1358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.557575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:44717 (local address 127.24.71.193:40163)
0811 22:26:04.558058 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.558075 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.558178 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.558574 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.558580 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.558600 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:04.558714 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.558718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.559473 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.559477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.560399 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.560409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.561083 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.561087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.561203 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.561220 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.561309 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.561378 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":29,"thread_start_us":109,"threads_started":1}
I20260811 22:26:04.561498 1357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.557547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:37369 (local address 127.24.71.193:51501)
0811 22:26:04.557935 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.557962 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.558080 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.558360 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.558366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.558389 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:04.558526 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.558535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.559309 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.559315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.560328 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.560339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.561012 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.561016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.561114 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.561145 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.561236 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.561310 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":49,"thread_start_us":127,"threads_started":1}
I20260811 22:26:04.562215 1109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ac4d84e93c4d09aa2a196a24bed646" candidate_uuid: "82e38521232d4cccb66debda5a10f7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08991788adaa4e9cabe6706a70e3b6f3" is_pre_election: true
I20260811 22:26:04.562558 1109 raft_consensus.cc:2454] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82e38521232d4cccb66debda5a10f7bb in term 0.
I20260811 22:26:04.562567 1266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ac4d84e93c4d09aa2a196a24bed646" candidate_uuid: "82e38521232d4cccb66debda5a10f7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5e5d5a5395349f6adc7ed239db856c7" is_pre_election: true
I20260811 22:26:04.562815 1266 raft_consensus.cc:2454] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82e38521232d4cccb66debda5a10f7bb in term 0.
I20260811 22:26:04.563060 877 leader_election.cc:302] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08991788adaa4e9cabe6706a70e3b6f3, 82e38521232d4cccb66debda5a10f7bb; no voters:
I20260811 22:26:04.563390 1355 raft_consensus.cc:2781] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:26:04.563486 1355 raft_consensus.cc:482] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:26:04.563542 1355 raft_consensus.cc:3037] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:04.564988 1355 raft_consensus.cc:504] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } }
I20260811 22:26:04.565388 1355 leader_election.cc:288] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [CANDIDATE]: Term 1 election: Requested vote from peers c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369), 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717)
I20260811 22:26:04.565754 1266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ac4d84e93c4d09aa2a196a24bed646" candidate_uuid: "82e38521232d4cccb66debda5a10f7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5e5d5a5395349f6adc7ed239db856c7"
I20260811 22:26:04.565755 1109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ac4d84e93c4d09aa2a196a24bed646" candidate_uuid: "82e38521232d4cccb66debda5a10f7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08991788adaa4e9cabe6706a70e3b6f3"
I20260811 22:26:04.565886 1266 raft_consensus.cc:3037] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:04.565886 1109 raft_consensus.cc:3037] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:04.567240 1266 raft_consensus.cc:2454] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82e38521232d4cccb66debda5a10f7bb in term 1.
I20260811 22:26:04.567240 1109 raft_consensus.cc:2454] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82e38521232d4cccb66debda5a10f7bb in term 1.
I20260811 22:26:04.567591 876 leader_election.cc:302] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82e38521232d4cccb66debda5a10f7bb, c5e5d5a5395349f6adc7ed239db856c7; no voters:
I20260811 22:26:04.567796 1355 raft_consensus.cc:2781] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:04.568053 1355 raft_consensus.cc:686] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 1 LEADER]: Becoming Leader. State: Replica: 82e38521232d4cccb66debda5a10f7bb, State: Running, Role: LEADER
I20260811 22:26:04.568389 1355 consensus_queue.cc:237] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } }
I20260811 22:26:04.571928 780 catalog_manager.cc:5495] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 82e38521232d4cccb66debda5a10f7bb (127.24.71.193). New cstate: current_term: 1 leader_uuid: "82e38521232d4cccb66debda5a10f7bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } health_report { overall_health: UNKNOWN } } }
I20260811 22:26:04.585743 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.582064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39656 (local address 127.24.71.193:40637)
0811 22:26:04.582207 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.582212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.582227 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:04.582390 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.582392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.582454 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.582570 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:04.582814 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.583313 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.584252 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.584490 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.584601 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.584892 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:04.585334 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:04.585470 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.585534 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.585594 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:26:04.591691 1266 raft_consensus.cc:1261] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Refusing update from remote peer 82e38521232d4cccb66debda5a10f7bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:04.592208 1109 raft_consensus.cc:1261] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Refusing update from remote peer 82e38521232d4cccb66debda5a10f7bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:04.592702 1355 consensus_queue.cc:1035] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:04.592911 1359 consensus_queue.cc:1035] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:04.602061 1362 mvcc.cc:205] Tried to move back new op lower bound from 7317451426160439296 to 7317451426075119616. Current Snapshot: MvccSnapshot[applied={T|T < 7317451426160439296}]
I20260811 22:26:04.607335 1364 mvcc.cc:205] Tried to move back new op lower bound from 7317451426160439296 to 7317451426075119616. Current Snapshot: MvccSnapshot[applied={T|T < 7317451426160439296}]
I20260811 22:26:04.611186 781 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:04.611413 781 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:04.614283 781 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564: Sending DeleteTablet for 3 replicas of tablet 21ac4d84e93c4d09aa2a196a24bed646
I20260811 22:26:04.615100 927 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ac4d84e93c4d09aa2a196a24bed646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:04 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:39642
I20260811 22:26:04.615381 1087 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ac4d84e93c4d09aa2a196a24bed646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:04 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60152
I20260811 22:26:04.615741 1372 tablet_replica.cc:330] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: stopping tablet replica
I20260811 22:26:04.615911 1245 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ac4d84e93c4d09aa2a196a24bed646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:04 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58180
I20260811 22:26:04.616417 1374 tablet_replica.cc:330] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: stopping tablet replica
I20260811 22:26:04.616465 1373 tablet_replica.cc:330] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: stopping tablet replica
I20260811 22:26:04.616779 1373 raft_consensus.cc:2229] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:04.616779 1374 raft_consensus.cc:2229] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:04.616745 781 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:26:04.617125 1373 raft_consensus.cc:2258] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 22:26:04.617175 781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:26:04.617689 1374 raft_consensus.cc:2258] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:04.618142 1372 raft_consensus.cc:2229] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:26:04.618538 1372 raft_consensus.cc:2258] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:04.619519 1373 ts_tablet_manager.cc:1905] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:04.620632 1372 ts_tablet_manager.cc:1905] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:04.622002 1374 ts_tablet_manager.cc:1905] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:04.622516 927 tablet_service.cc:1467] Processing CreateTablet for tablet ced1f545975f40f1afeb3d54196efde7 (DEFAULT_TABLE table=test-table [id=c632746038a84f0ca9dbf1ed3d8016fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:04.622782 1245 tablet_service.cc:1467] Processing CreateTablet for tablet ced1f545975f40f1afeb3d54196efde7 (DEFAULT_TABLE table=test-table [id=c632746038a84f0ca9dbf1ed3d8016fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:04.622835 927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ced1f545975f40f1afeb3d54196efde7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:04.623055 1245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ced1f545975f40f1afeb3d54196efde7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:04.625406 1349 tablet_bootstrap.cc:492] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: Bootstrap starting.
I20260811 22:26:04.625501 1350 tablet_bootstrap.cc:492] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: Bootstrap starting.
I20260811 22:26:04.626389 1349 tablet_bootstrap.cc:654] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:04.626511 1350 tablet_bootstrap.cc:654] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:04.627575 1372 ts_tablet_manager.cc:1918] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:26:04.627678 1372 log.cc:1196] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/21ac4d84e93c4d09aa2a196a24bed646
I20260811 22:26:04.627715 1349 tablet_bootstrap.cc:492] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: No bootstrap required, opened a new log
I20260811 22:26:04.627794 1349 ts_tablet_manager.cc:1397] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:26:04.627883 1087 tablet_service.cc:1467] Processing CreateTablet for tablet ced1f545975f40f1afeb3d54196efde7 (DEFAULT_TABLE table=test-table [id=c632746038a84f0ca9dbf1ed3d8016fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:04.628062 1372 ts_tablet_manager.cc:1939] T 21ac4d84e93c4d09aa2a196a24bed646 P 82e38521232d4cccb66debda5a10f7bb: Deleting consensus metadata
I20260811 22:26:04.628185 1087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ced1f545975f40f1afeb3d54196efde7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:04.628299 1349 raft_consensus.cc:348] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.628404 1349 raft_consensus.cc:374] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:04.628444 1349 raft_consensus.cc:729] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e38521232d4cccb66debda5a10f7bb, State: Initialized, Role: FOLLOWER
I20260811 22:26:04.628589 1349 consensus_queue.cc:260] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.628839 1349 ts_tablet_manager.cc:1428] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:26:04.630546 1347 tablet_bootstrap.cc:492] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: Bootstrap starting.
I20260811 22:26:04.631559 1347 tablet_bootstrap.cc:654] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:04.632351 1374 ts_tablet_manager.cc:1918] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:26:04.632396 1373 ts_tablet_manager.cc:1918] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:26:04.632479 1374 log.cc:1196] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/21ac4d84e93c4d09aa2a196a24bed646
I20260811 22:26:04.632498 1373 log.cc:1196] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/21ac4d84e93c4d09aa2a196a24bed646
I20260811 22:26:04.632827 1373 ts_tablet_manager.cc:1939] T 21ac4d84e93c4d09aa2a196a24bed646 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting consensus metadata
I20260811 22:26:04.633019 1347 tablet_bootstrap.cc:492] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: No bootstrap required, opened a new log
I20260811 22:26:04.633095 1347 ts_tablet_manager.cc:1397] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:04.633527 1347 raft_consensus.cc:348] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.633633 1347 raft_consensus.cc:374] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:04.633649 1374 ts_tablet_manager.cc:1939] T 21ac4d84e93c4d09aa2a196a24bed646 P c5e5d5a5395349f6adc7ed239db856c7: Deleting consensus metadata
I20260811 22:26:04.633662 1347 raft_consensus.cc:729] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08991788adaa4e9cabe6706a70e3b6f3, State: Initialized, Role: FOLLOWER
I20260811 22:26:04.633790 1347 consensus_queue.cc:260] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.634297 767 catalog_manager.cc:4841] TS 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717): tablet 21ac4d84e93c4d09aa2a196a24bed646 (table test-table [id=1b39e87a3a2847ddbe248b57d72fe7a8]) successfully deleted
I20260811 22:26:04.634807 1350 tablet_bootstrap.cc:492] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: No bootstrap required, opened a new log
I20260811 22:26:04.634902 1350 ts_tablet_manager.cc:1397] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260811 22:26:04.634856 766 catalog_manager.cc:4841] TS c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369): tablet 21ac4d84e93c4d09aa2a196a24bed646 (table test-table [id=1b39e87a3a2847ddbe248b57d72fe7a8]) successfully deleted
I20260811 22:26:04.635378 765 catalog_manager.cc:4841] TS 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637): tablet 21ac4d84e93c4d09aa2a196a24bed646 (table test-table [id=1b39e87a3a2847ddbe248b57d72fe7a8]) successfully deleted
I20260811 22:26:04.635377 1350 raft_consensus.cc:348] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.635502 1350 raft_consensus.cc:374] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:04.635540 1350 raft_consensus.cc:729] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e5d5a5395349f6adc7ed239db856c7, State: Initialized, Role: FOLLOWER
I20260811 22:26:04.635663 1350 consensus_queue.cc:260] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.635991 1350 ts_tablet_manager.cc:1428] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:26:04.636134 1347 ts_tablet_manager.cc:1428] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20260811 22:26:04.694856 1003 tablet.cc:2369] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:26:04.702793 1317 tablet.cc:2369] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:26:04.706866 1161 tablet.cc:2369] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:26:04.740108 1355 raft_consensus.cc:482] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:26:04.740317 1355 raft_consensus.cc:504] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.740903 1355 leader_election.cc:288] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717), c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369)
I20260811 22:26:04.741415 1109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced1f545975f40f1afeb3d54196efde7" candidate_uuid: "82e38521232d4cccb66debda5a10f7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08991788adaa4e9cabe6706a70e3b6f3" is_pre_election: true
I20260811 22:26:04.741469 1266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced1f545975f40f1afeb3d54196efde7" candidate_uuid: "82e38521232d4cccb66debda5a10f7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5e5d5a5395349f6adc7ed239db856c7" is_pre_election: true
I20260811 22:26:04.741600 1109 raft_consensus.cc:2454] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82e38521232d4cccb66debda5a10f7bb in term 0.
I20260811 22:26:04.741609 1266 raft_consensus.cc:2454] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82e38521232d4cccb66debda5a10f7bb in term 0.
I20260811 22:26:04.741940 876 leader_election.cc:302] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82e38521232d4cccb66debda5a10f7bb, c5e5d5a5395349f6adc7ed239db856c7; no voters:
I20260811 22:26:04.742163 1355 raft_consensus.cc:2781] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:26:04.742230 1355 raft_consensus.cc:482] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:26:04.742285 1355 raft_consensus.cc:3037] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:04.743309 1355 raft_consensus.cc:504] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.743714 1355 leader_election.cc:288] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [CANDIDATE]: Term 1 election: Requested vote from peers 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717), c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369)
I20260811 22:26:04.744159 1109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced1f545975f40f1afeb3d54196efde7" candidate_uuid: "82e38521232d4cccb66debda5a10f7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08991788adaa4e9cabe6706a70e3b6f3"
I20260811 22:26:04.744166 1266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ced1f545975f40f1afeb3d54196efde7" candidate_uuid: "82e38521232d4cccb66debda5a10f7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5e5d5a5395349f6adc7ed239db856c7"
I20260811 22:26:04.744307 1109 raft_consensus.cc:3037] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:04.744326 1266 raft_consensus.cc:3037] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:04.745457 1109 raft_consensus.cc:2454] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82e38521232d4cccb66debda5a10f7bb in term 1.
I20260811 22:26:04.745586 1266 raft_consensus.cc:2454] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82e38521232d4cccb66debda5a10f7bb in term 1.
I20260811 22:26:04.745932 876 leader_election.cc:302] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82e38521232d4cccb66debda5a10f7bb, c5e5d5a5395349f6adc7ed239db856c7; no voters:
I20260811 22:26:04.746148 1355 raft_consensus.cc:2781] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:04.746275 1355 raft_consensus.cc:686] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 1 LEADER]: Becoming Leader. State: Replica: 82e38521232d4cccb66debda5a10f7bb, State: Running, Role: LEADER
I20260811 22:26:04.746634 1355 consensus_queue.cc:237] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.748590 780 catalog_manager.cc:5495] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 82e38521232d4cccb66debda5a10f7bb (127.24.71.193). New cstate: current_term: 1 leader_uuid: "82e38521232d4cccb66debda5a10f7bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:26:04 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487163, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.254@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:26:04.780360 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.769958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56044 (local address 127.24.71.254:36661)
0811 22:26:04.770095 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.770100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.770308 (+ 208us) server_negotiation.cc:407] Connection header received
0811 22:26:04.770592 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.770595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.770643 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.770720 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:04.771579 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.772236 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.773432 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.773669 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.773764 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.776675 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:04.776678 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:04.776681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:04.776704 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:04.778350 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.778895 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.778901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.778903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.778991 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:04.779303 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:04.779307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:04.779309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:04.779515 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:04.779635 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.780007 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.780223 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":55}
I20260811 22:26:04.780362 1384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.769800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36661 (local address 127.0.0.1:56044)
0811 22:26:04.770173 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.770189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.770317 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.770778 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.770781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.771308 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:04.771430 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.771438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.772344 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.772348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.773278 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.773289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.773792 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.773798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.773894 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.774578 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:04.774594 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:04.776507 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:04.778502 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.778514 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.778521 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.778780 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.779100 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:04.779103 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:04.779105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:04.779199 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:04.779635 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:04.779644 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:04.779771 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.779952 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.780136 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":135,"threads_started":1}
I20260811 22:26:04.784947 780 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:56044:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:26:04.785729 780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:26:04.791322 1245 tablet_service.cc:1467] Processing CreateTablet for tablet 16403347a40543ca894dd8174b7c355d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8418b189931d42afaaf1f85644044063]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:26:04.791322 927 tablet_service.cc:1467] Processing CreateTablet for tablet 16403347a40543ca894dd8174b7c355d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8418b189931d42afaaf1f85644044063]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:26:04.791476 1087 tablet_service.cc:1467] Processing CreateTablet for tablet 16403347a40543ca894dd8174b7c355d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8418b189931d42afaaf1f85644044063]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:26:04.791745 927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16403347a40543ca894dd8174b7c355d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:04.791746 1245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16403347a40543ca894dd8174b7c355d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:04.791925 1087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16403347a40543ca894dd8174b7c355d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:04.794682 1349 tablet_bootstrap.cc:492] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: Bootstrap starting.
I20260811 22:26:04.794688 1350 tablet_bootstrap.cc:492] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: Bootstrap starting.
I20260811 22:26:04.794879 1347 tablet_bootstrap.cc:492] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: Bootstrap starting.
I20260811 22:26:04.795522 1350 tablet_bootstrap.cc:654] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:04.795571 1349 tablet_bootstrap.cc:654] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:04.795778 1347 tablet_bootstrap.cc:654] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:04.796793 1350 tablet_bootstrap.cc:492] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: No bootstrap required, opened a new log
I20260811 22:26:04.796902 1350 ts_tablet_manager.cc:1397] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:26:04.797338 1347 tablet_bootstrap.cc:492] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: No bootstrap required, opened a new log
I20260811 22:26:04.797438 1347 ts_tablet_manager.cc:1397] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:04.797389 1350 raft_consensus.cc:348] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.797502 1350 raft_consensus.cc:374] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:04.797551 1350 raft_consensus.cc:729] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e5d5a5395349f6adc7ed239db856c7, State: Initialized, Role: FOLLOWER
I20260811 22:26:04.797688 1350 consensus_queue.cc:260] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.797887 1347 raft_consensus.cc:348] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.797979 1350 ts_tablet_manager.cc:1428] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:26:04.798002 1347 raft_consensus.cc:374] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:04.798036 1347 raft_consensus.cc:729] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08991788adaa4e9cabe6706a70e3b6f3, State: Initialized, Role: FOLLOWER
I20260811 22:26:04.798168 1347 consensus_queue.cc:260] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.798189 1354 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.798372 1354 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:26:04.798477 1347 ts_tablet_manager.cc:1428] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:26:04.799072 1356 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.799158 1349 tablet_bootstrap.cc:492] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: No bootstrap required, opened a new log
I20260811 22:26:04.799216 1356 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:26:04.799249 1349 ts_tablet_manager.cc:1397] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260811 22:26:04.799671 1349 raft_consensus.cc:348] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.799813 1349 raft_consensus.cc:374] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:04.799868 1349 raft_consensus.cc:729] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e38521232d4cccb66debda5a10f7bb, State: Initialized, Role: FOLLOWER
I20260811 22:26:04.799998 1349 consensus_queue.cc:260] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.800263 1349 ts_tablet_manager.cc:1428] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:26:04.800271 1355 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.800469 1355 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:26:04.918516 1354 raft_consensus.cc:482] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:26:04.918700 1354 raft_consensus.cc:504] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.919977 1354 leader_election.cc:288] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637), 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717)
I20260811 22:26:04.924438 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.920398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:47831 (local address 127.24.71.193:40637)
0811 22:26:04.920683 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.920688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.920705 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:26:04.920769 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.920772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.920819 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.920913 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:04.921233 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.921758 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.923143 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.923758 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.923918 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.923984 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.924239 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.924311 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
I20260811 22:26:04.924438 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.920266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40637 (local address 127.24.71.195:47831)
0811 22:26:04.920509 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.920527 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.920628 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.920934 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.920937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.920953 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:04.921104 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.921112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.921878 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.921884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.923001 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.923012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.923997 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.924004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.924104 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.924123 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.924218 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.924293 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":1}
I20260811 22:26:04.924599 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.920407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:35197 (local address 127.24.71.194:44717)
0811 22:26:04.920539 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.920546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.920755 (+ 209us) server_negotiation.cc:407] Connection header received
0811 22:26:04.920912 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.920914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.920982 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.921078 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:04.921739 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.922359 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.923446 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.924042 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.924175 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.924226 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.924413 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.924484 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:26:04.924620 1388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.920266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:44717 (local address 127.24.71.195:35197)
0811 22:26:04.920645 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.920664 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.920796 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.921100 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.921103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.921121 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:04.921606 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.921614 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.922476 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.922481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.923320 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.923329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.924170 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.924174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.924289 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.924307 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.924397 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.924474 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":68,"thread_start_us":77,"threads_started":1}
I20260811 22:26:04.925030 950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16403347a40543ca894dd8174b7c355d" candidate_uuid: "c5e5d5a5395349f6adc7ed239db856c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82e38521232d4cccb66debda5a10f7bb" is_pre_election: true
I20260811 22:26:04.925308 950 raft_consensus.cc:2454] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5e5d5a5395349f6adc7ed239db856c7 in term 0.
I20260811 22:26:04.925455 1109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16403347a40543ca894dd8174b7c355d" candidate_uuid: "c5e5d5a5395349f6adc7ed239db856c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08991788adaa4e9cabe6706a70e3b6f3" is_pre_election: true
I20260811 22:26:04.925621 1109 raft_consensus.cc:2454] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5e5d5a5395349f6adc7ed239db856c7 in term 0.
I20260811 22:26:04.925813 1189 leader_election.cc:302] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82e38521232d4cccb66debda5a10f7bb, c5e5d5a5395349f6adc7ed239db856c7; no voters:
I20260811 22:26:04.926108 1354 raft_consensus.cc:2781] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:26:04.926163 1354 raft_consensus.cc:482] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:26:04.926191 1354 raft_consensus.cc:3037] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:04.927196 1354 raft_consensus.cc:504] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.927580 1354 leader_election.cc:288] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [CANDIDATE]: Term 1 election: Requested vote from peers 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637), 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717)
I20260811 22:26:04.927966 1109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16403347a40543ca894dd8174b7c355d" candidate_uuid: "c5e5d5a5395349f6adc7ed239db856c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08991788adaa4e9cabe6706a70e3b6f3"
I20260811 22:26:04.928000 950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16403347a40543ca894dd8174b7c355d" candidate_uuid: "c5e5d5a5395349f6adc7ed239db856c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82e38521232d4cccb66debda5a10f7bb"
I20260811 22:26:04.928105 1109 raft_consensus.cc:3037] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:04.928134 950 raft_consensus.cc:3037] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:04.929080 1109 raft_consensus.cc:2454] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5e5d5a5395349f6adc7ed239db856c7 in term 1.
I20260811 22:26:04.929167 950 raft_consensus.cc:2454] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5e5d5a5395349f6adc7ed239db856c7 in term 1.
I20260811 22:26:04.929430 1191 leader_election.cc:302] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08991788adaa4e9cabe6706a70e3b6f3, c5e5d5a5395349f6adc7ed239db856c7; no voters:
I20260811 22:26:04.929661 1354 raft_consensus.cc:2781] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:04.929894 1354 raft_consensus.cc:686] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [term 1 LEADER]: Becoming Leader. State: Replica: c5e5d5a5395349f6adc7ed239db856c7, State: Running, Role: LEADER
I20260811 22:26:04.930181 1354 consensus_queue.cc:237] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:04.931425 1389 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: TxnStatusTablet state changed. Reason: New leader c5e5d5a5395349f6adc7ed239db856c7. Latest consensus state: current_term: 1 leader_uuid: "c5e5d5a5395349f6adc7ed239db856c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.931541 1389 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:26:04.931885 1391 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:26:04.932066 1391 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:26:04.932854 777 catalog_manager.cc:5495] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "c5e5d5a5395349f6adc7ed239db856c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } health_report { overall_health: HEALTHY } } }
I20260811 22:26:04.959255 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.954280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.24.71.195:37369)
0811 22:26:04.954471 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.954476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.954493 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:26:04.954577 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.954580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.954652 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.954757 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:04.955069 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.955889 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.957221 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.957580 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.957694 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.958227 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:04.958815 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:04.958931 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.959039 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.959101 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:26:04.959276 1384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.953574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:37369 (local address 127.0.0.1:58186)
0811 22:26:04.953955 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.953972 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.954094 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.954809 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.954813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.954822 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:26:04.954941 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.954948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.956008 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.956012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.957077 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.957090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.957900 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.957904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.958003 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.958071 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:26:04.958921 (+ 850us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:26:04.958942 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.959027 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.959130 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259}
I20260811 22:26:04.962896 1109 raft_consensus.cc:1261] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Refusing update from remote peer c5e5d5a5395349f6adc7ed239db856c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:04.963071 950 raft_consensus.cc:1261] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Refusing update from remote peer c5e5d5a5395349f6adc7ed239db856c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:04.963442 1354 consensus_queue.cc:1035] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:04.963660 1389 consensus_queue.cc:1035] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:04.965148 1356 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: TxnStatusTablet state changed. Reason: New leader c5e5d5a5395349f6adc7ed239db856c7. Latest consensus state: current_term: 1 leader_uuid: "c5e5d5a5395349f6adc7ed239db856c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.965309 1356 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:26:04.966064 1355 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: TxnStatusTablet state changed. Reason: New leader c5e5d5a5395349f6adc7ed239db856c7. Latest consensus state: current_term: 1 leader_uuid: "c5e5d5a5395349f6adc7ed239db856c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.966207 1355 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:26:04.967072 1354 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5e5d5a5395349f6adc7ed239db856c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.967221 1354 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:26:04.967314 1363 mvcc.cc:205] Tried to move back new op lower bound from 7317451427681841152 to 7317451427556642816. Current Snapshot: MvccSnapshot[applied={T|T < 7317451427681841152}]
I20260811 22:26:04.967855 1356 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5e5d5a5395349f6adc7ed239db856c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.967968 1356 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P 08991788adaa4e9cabe6706a70e3b6f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:26:04.973423 1389 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5e5d5a5395349f6adc7ed239db856c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.973577 1389 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P c5e5d5a5395349f6adc7ed239db856c7: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:26:04.974189 1355 tablet_replica.cc:439] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5e5d5a5395349f6adc7ed239db856c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } } }
I20260811 22:26:04.974309 1355 tablet_replica.cc:442] T 16403347a40543ca894dd8174b7c355d P 82e38521232d4cccb66debda5a10f7bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:26:04.985599 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.981537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:37369 (local address 127.24.71.193:49987)
0811 22:26:04.981875 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:04.981888 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:04.981994 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:04.982245 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:04.982248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:04.982255 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:26:04.982353 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.982357 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.983401 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.983405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.984141 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:04.984152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.984773 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.984776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:04.984850 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.984904 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:26:04.985276 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:26:04.985284 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:26:04.985350 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:26:04.985435 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":137,"threads_started":1}
I20260811 22:26:04.985602 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:04.981647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:49987 (local address 127.24.71.195:37369)
0811 22:26:04.981770 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:04.981775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:04.981969 (+ 194us) server_negotiation.cc:407] Connection header received
0811 22:26:04.982078 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:04.982081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:04.982136 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:04.982221 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:04.982472 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.983259 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.984269 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:04.984625 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:04.984733 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:04.985065 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:04.985156 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:04.985232 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:04.985390 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 22:26:04.985461 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:26:04.992832 1266 raft_consensus.cc:1261] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Refusing update from remote peer 82e38521232d4cccb66debda5a10f7bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:04.992852 1109 raft_consensus.cc:1261] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Refusing update from remote peer 82e38521232d4cccb66debda5a10f7bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:04.993275 1359 consensus_queue.cc:1035] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:04.993494 1355 consensus_queue.cc:1035] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:05.018862 1407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.014152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40637 (local address 127.24.71.195:57129)
0811 22:26:05.014520 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:05.014534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:05.014637 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:05.014911 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:05.014914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:05.014923 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:26:05.015054 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.015060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.016143 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.016147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.017456 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.017469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.018003 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.018007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.018109 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.018177 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.018506 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.018512 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:26:05.018579 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:26:05.018669 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":141,"threads_started":1}
I20260811 22:26:05.018890 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.014206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:57129 (local address 127.24.71.193:40637)
0811 22:26:05.014332 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.014337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.014608 (+ 271us) server_negotiation.cc:407] Connection header received
0811 22:26:05.014716 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.014719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.014775 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.014858 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:05.015180 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.016002 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.017612 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.017878 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.017977 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.018344 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.018437 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.018501 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.018609 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.018705 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:26:05.047222 777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:05.047384 777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:05.050318 777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564: Sending DeleteTablet for 3 replicas of tablet ced1f545975f40f1afeb3d54196efde7
I20260811 22:26:05.051182 1087 tablet_service.cc:1514] Processing DeleteTablet for tablet ced1f545975f40f1afeb3d54196efde7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:05 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60152
I20260811 22:26:05.051678 1373 tablet_replica.cc:330] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: stopping tablet replica
I20260811 22:26:05.051043 927 tablet_service.cc:1514] Processing DeleteTablet for tablet ced1f545975f40f1afeb3d54196efde7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:05 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:39642
I20260811 22:26:05.051929 1372 tablet_replica.cc:330] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: stopping tablet replica
I20260811 22:26:05.051927 1245 tablet_service.cc:1514] Processing DeleteTablet for tablet ced1f545975f40f1afeb3d54196efde7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:05 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58180
I20260811 22:26:05.052060 1372 raft_consensus.cc:2229] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:26:05.052047 1373 raft_consensus.cc:2229] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:05.052215 1373 raft_consensus.cc:2258] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:05.052311 1372 raft_consensus.cc:2258] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:05.052309 1374 tablet_replica.cc:330] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: stopping tablet replica
I20260811 22:26:05.052593 1374 raft_consensus.cc:2229] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:05.052841 1374 raft_consensus.cc:2258] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:05.053691 1372 ts_tablet_manager.cc:1905] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:05.054029 1373 ts_tablet_manager.cc:1905] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:05.054271 1374 ts_tablet_manager.cc:1905] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:05.057163 1372 ts_tablet_manager.cc:1918] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:26:05.057401 1373 ts_tablet_manager.cc:1918] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:26:05.057487 1373 log.cc:1196] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/ced1f545975f40f1afeb3d54196efde7
I20260811 22:26:05.057785 1373 ts_tablet_manager.cc:1939] T ced1f545975f40f1afeb3d54196efde7 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting consensus metadata
I20260811 22:26:05.058766 1372 log.cc:1196] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/ced1f545975f40f1afeb3d54196efde7
I20260811 22:26:05.058847 767 catalog_manager.cc:4841] TS 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717): tablet ced1f545975f40f1afeb3d54196efde7 (table test-table [id=c632746038a84f0ca9dbf1ed3d8016fc]) successfully deleted
I20260811 22:26:05.059093 1374 ts_tablet_manager.cc:1918] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:26:05.059203 1374 log.cc:1196] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/ced1f545975f40f1afeb3d54196efde7
I20260811 22:26:05.059352 1372 ts_tablet_manager.cc:1939] T ced1f545975f40f1afeb3d54196efde7 P 82e38521232d4cccb66debda5a10f7bb: Deleting consensus metadata
I20260811 22:26:05.059463 1374 ts_tablet_manager.cc:1939] T ced1f545975f40f1afeb3d54196efde7 P c5e5d5a5395349f6adc7ed239db856c7: Deleting consensus metadata
I20260811 22:26:05.060254 765 catalog_manager.cc:4841] TS 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637): tablet ced1f545975f40f1afeb3d54196efde7 (table test-table [id=c632746038a84f0ca9dbf1ed3d8016fc]) successfully deleted
I20260811 22:26:05.060319 766 catalog_manager.cc:4841] TS c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369): tablet ced1f545975f40f1afeb3d54196efde7 (table test-table [id=c632746038a84f0ca9dbf1ed3d8016fc]) successfully deleted
Aug 11 22:26:05 dist-test-slave-mvvj krb5kdc[732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:26:05 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:05 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:26:05.086608 835 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:05.079402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.24.71.254:36661)
0811 22:26:05.079562 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.079567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.079721 (+ 154us) server_negotiation.cc:407] Connection header received
0811 22:26:05.079927 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.079931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.080004 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.080115 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:05.080848 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.081659 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.082535 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.082869 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.083009 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.086464 (+ 3455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56058: BlockingRecv error: recv got EOF from 127.0.0.1:56058 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 22:26:05 dist-test-slave-mvvj krb5kdc[732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM
I20260811 22:26:05.098110 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.088501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56072 (local address 127.24.71.254:36661)
0811 22:26:05.088629 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.088633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.088821 (+ 188us) server_negotiation.cc:407] Connection header received
0811 22:26:05.089015 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.089018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.089074 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.089163 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:05.089733 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.090536 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.091335 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.091661 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.091776 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.094555 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:05.094559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:05.094561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:05.094585 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:05.096284 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:05.096819 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:05.096822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:05.096824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:05.096903 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:05.097232 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:05.097234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:05.097235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:05.097412 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:05.097528 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.097802 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.097903 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:26:05.101192 777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:26:05.101748 777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:26:05.107084 1245 tablet_service.cc:1467] Processing CreateTablet for tablet 737ee195226b48609bcb05eafb814384 (DEFAULT_TABLE table=test-table [id=4c843e558ee84ca99578038cd77725fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:05.107084 927 tablet_service.cc:1467] Processing CreateTablet for tablet 737ee195226b48609bcb05eafb814384 (DEFAULT_TABLE table=test-table [id=4c843e558ee84ca99578038cd77725fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:05.107431 1245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 737ee195226b48609bcb05eafb814384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:05.107431 927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 737ee195226b48609bcb05eafb814384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:05.109998 1350 tablet_bootstrap.cc:492] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: Bootstrap starting.
I20260811 22:26:05.110143 1087 tablet_service.cc:1467] Processing CreateTablet for tablet 737ee195226b48609bcb05eafb814384 (DEFAULT_TABLE table=test-table [id=4c843e558ee84ca99578038cd77725fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:05.110468 1087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 737ee195226b48609bcb05eafb814384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:05.111014 1350 tablet_bootstrap.cc:654] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:05.112548 1350 tablet_bootstrap.cc:492] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: No bootstrap required, opened a new log
I20260811 22:26:05.112653 1350 ts_tablet_manager.cc:1397] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:05.112962 1347 tablet_bootstrap.cc:492] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: Bootstrap starting.
I20260811 22:26:05.113117 1350 raft_consensus.cc:348] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.113255 1350 raft_consensus.cc:374] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:05.113288 1350 raft_consensus.cc:729] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e5d5a5395349f6adc7ed239db856c7, State: Initialized, Role: FOLLOWER
I20260811 22:26:05.113598 1350 consensus_queue.cc:260] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.113996 1347 tablet_bootstrap.cc:654] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:05.114580 1350 ts_tablet_manager.cc:1428] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:26:05.114809 1349 tablet_bootstrap.cc:492] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: Bootstrap starting.
I20260811 22:26:05.115773 1349 tablet_bootstrap.cc:654] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:05.117514 1347 tablet_bootstrap.cc:492] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: No bootstrap required, opened a new log
I20260811 22:26:05.117643 1347 ts_tablet_manager.cc:1397] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:26:05.117957 1349 tablet_bootstrap.cc:492] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: No bootstrap required, opened a new log
I20260811 22:26:05.118081 1349 ts_tablet_manager.cc:1397] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:26:05.118116 1347 raft_consensus.cc:348] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.118271 1347 raft_consensus.cc:374] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:05.118327 1347 raft_consensus.cc:729] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08991788adaa4e9cabe6706a70e3b6f3, State: Initialized, Role: FOLLOWER
I20260811 22:26:05.118485 1347 consensus_queue.cc:260] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.118590 1349 raft_consensus.cc:348] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.118742 1349 raft_consensus.cc:374] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:05.118783 1349 raft_consensus.cc:729] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e38521232d4cccb66debda5a10f7bb, State: Initialized, Role: FOLLOWER
I20260811 22:26:05.118839 1347 ts_tablet_manager.cc:1428] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:26:05.118927 1349 consensus_queue.cc:260] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.119223 1349 ts_tablet_manager.cc:1428] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:26:05.365347 1356 raft_consensus.cc:482] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:26:05.365563 1356 raft_consensus.cc:504] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.366799 1356 leader_election.cc:288] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637), c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369)
I20260811 22:26:05.371079 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.367277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:42089 (local address 127.24.71.193:40637)
0811 22:26:05.367428 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.367432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.367699 (+ 267us) server_negotiation.cc:407] Connection header received
0811 22:26:05.367896 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.367898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.367946 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.368033 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:05.368311 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.368816 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.369999 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.370403 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.370538 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.370600 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.370848 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.370936 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:26:05.371081 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.367219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:54999 (local address 127.24.71.195:37369)
0811 22:26:05.367360 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.367366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.367625 (+ 259us) server_negotiation.cc:407] Connection header received
0811 22:26:05.367803 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.367809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.367918 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.368039 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:05.368309 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.368812 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.369997 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.370418 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.370554 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.370613 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.370849 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.370931 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:26:05.371107 1427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.367124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40637 (local address 127.24.71.194:42089)
0811 22:26:05.367599 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:05.367612 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:05.367739 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:05.368043 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:05.368046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:05.368062 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:05.368199 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.368204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.368936 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.368942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.369866 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.369876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.370537 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.370541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.370703 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.370731 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:26:05.370822 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:26:05.370950 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":116,"thread_start_us":79,"threads_started":1}
I20260811 22:26:05.371117 1426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.367111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:37369 (local address 127.24.71.194:54999)
0811 22:26:05.367493 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:05.367506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:05.367681 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:05.368033 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:05.368036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:05.368054 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:05.368185 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.368197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.368933 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.368938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.369860 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.369870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.370554 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.370558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.370712 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.370736 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:26:05.370827 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:26:05.370940 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":83,"spinlock_wait_cycles":14592,"thread_start_us":88,"threads_started":1}
I20260811 22:26:05.371551 950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737ee195226b48609bcb05eafb814384" candidate_uuid: "08991788adaa4e9cabe6706a70e3b6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82e38521232d4cccb66debda5a10f7bb" is_pre_election: true
I20260811 22:26:05.371573 1266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737ee195226b48609bcb05eafb814384" candidate_uuid: "08991788adaa4e9cabe6706a70e3b6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5e5d5a5395349f6adc7ed239db856c7" is_pre_election: true
I20260811 22:26:05.371716 950 raft_consensus.cc:2454] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08991788adaa4e9cabe6706a70e3b6f3 in term 0.
I20260811 22:26:05.371733 1266 raft_consensus.cc:2454] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08991788adaa4e9cabe6706a70e3b6f3 in term 0.
I20260811 22:26:05.372177 1033 leader_election.cc:302] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08991788adaa4e9cabe6706a70e3b6f3, 82e38521232d4cccb66debda5a10f7bb; no voters:
I20260811 22:26:05.372483 1356 raft_consensus.cc:2781] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:26:05.372546 1356 raft_consensus.cc:482] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:26:05.372586 1356 raft_consensus.cc:3037] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:05.373648 1356 raft_consensus.cc:504] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.374082 1356 leader_election.cc:288] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637), c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369)
I20260811 22:26:05.374492 950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737ee195226b48609bcb05eafb814384" candidate_uuid: "08991788adaa4e9cabe6706a70e3b6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82e38521232d4cccb66debda5a10f7bb"
I20260811 22:26:05.374564 1266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737ee195226b48609bcb05eafb814384" candidate_uuid: "08991788adaa4e9cabe6706a70e3b6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5e5d5a5395349f6adc7ed239db856c7"
I20260811 22:26:05.374658 950 raft_consensus.cc:3037] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:05.374679 1266 raft_consensus.cc:3037] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:05.375957 1266 raft_consensus.cc:2454] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08991788adaa4e9cabe6706a70e3b6f3 in term 1.
I20260811 22:26:05.375957 950 raft_consensus.cc:2454] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08991788adaa4e9cabe6706a70e3b6f3 in term 1.
I20260811 22:26:05.376294 1033 leader_election.cc:302] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08991788adaa4e9cabe6706a70e3b6f3, 82e38521232d4cccb66debda5a10f7bb; no voters:
I20260811 22:26:05.376508 1356 raft_consensus.cc:2781] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:05.376732 1356 raft_consensus.cc:686] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 08991788adaa4e9cabe6706a70e3b6f3, State: Running, Role: LEADER
I20260811 22:26:05.377013 1356 consensus_queue.cc:237] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.380143 780 catalog_manager.cc:5495] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194). New cstate: current_term: 1 leader_uuid: "08991788adaa4e9cabe6706a70e3b6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } health_report { overall_health: UNKNOWN } } }
I20260811 22:26:05.414098 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.410102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.24.71.194:44717)
0811 22:26:05.410251 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.410263 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.410279 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:26:05.410431 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.410433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.410491 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.410589 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:05.410826 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.411335 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.412246 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.412503 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.412587 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.412911 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.413468 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.413582 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.413748 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.413861 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:26:05.417325 1266 raft_consensus.cc:1261] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Refusing update from remote peer 08991788adaa4e9cabe6706a70e3b6f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:05.417376 950 raft_consensus.cc:1261] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Refusing update from remote peer 08991788adaa4e9cabe6706a70e3b6f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:05.417920 1356 consensus_queue.cc:1035] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:05.418242 1428 consensus_queue.cc:1035] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:05.431775 777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:05.432046 777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:05.434630 777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564: Sending DeleteTablet for 3 replicas of tablet 737ee195226b48609bcb05eafb814384
I20260811 22:26:05.435424 1087 tablet_service.cc:1514] Processing DeleteTablet for tablet 737ee195226b48609bcb05eafb814384 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:05 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60152
I20260811 22:26:05.435654 1373 tablet_replica.cc:330] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: stopping tablet replica
I20260811 22:26:05.435643 1245 tablet_service.cc:1514] Processing DeleteTablet for tablet 737ee195226b48609bcb05eafb814384 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:05 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58180
I20260811 22:26:05.435680 927 tablet_service.cc:1514] Processing DeleteTablet for tablet 737ee195226b48609bcb05eafb814384 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:05 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:39642
I20260811 22:26:05.435771 1373 raft_consensus.cc:2229] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:26:05.435910 1372 tablet_replica.cc:330] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: stopping tablet replica
I20260811 22:26:05.435920 1374 tablet_replica.cc:330] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: stopping tablet replica
I20260811 22:26:05.436038 1374 raft_consensus.cc:2229] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:05.436062 1372 raft_consensus.cc:2229] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:05.436168 1373 raft_consensus.cc:2258] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:05.436192 1374 raft_consensus.cc:2258] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:05.436257 1372 raft_consensus.cc:2258] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:05.437021 1374 ts_tablet_manager.cc:1905] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:05.437225 1373 ts_tablet_manager.cc:1905] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:05.437691 777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:26:05.438177 777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:26:05.439893 1374 ts_tablet_manager.cc:1918] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:26:05.439978 1374 log.cc:1196] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/737ee195226b48609bcb05eafb814384
I20260811 22:26:05.440251 1374 ts_tablet_manager.cc:1939] T 737ee195226b48609bcb05eafb814384 P c5e5d5a5395349f6adc7ed239db856c7: Deleting consensus metadata
I20260811 22:26:05.440851 1373 ts_tablet_manager.cc:1918] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:26:05.441075 1373 log.cc:1196] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/737ee195226b48609bcb05eafb814384
I20260811 22:26:05.441491 1372 ts_tablet_manager.cc:1905] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:05.441781 1373 ts_tablet_manager.cc:1939] T 737ee195226b48609bcb05eafb814384 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting consensus metadata
I20260811 22:26:05.442567 767 catalog_manager.cc:4841] TS 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717): tablet 737ee195226b48609bcb05eafb814384 (table test-table [id=4c843e558ee84ca99578038cd77725fc]) successfully deleted
I20260811 22:26:05.443995 1372 ts_tablet_manager.cc:1918] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:26:05.443981 766 catalog_manager.cc:4841] TS c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369): tablet 737ee195226b48609bcb05eafb814384 (table test-table [id=4c843e558ee84ca99578038cd77725fc]) successfully deleted
I20260811 22:26:05.444075 1372 log.cc:1196] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/737ee195226b48609bcb05eafb814384
I20260811 22:26:05.444387 1372 ts_tablet_manager.cc:1939] T 737ee195226b48609bcb05eafb814384 P 82e38521232d4cccb66debda5a10f7bb: Deleting consensus metadata
I20260811 22:26:05.448186 927 tablet_service.cc:1467] Processing CreateTablet for tablet 98a2caaab97d477a98bec54a442452d6 (DEFAULT_TABLE table=test-table [id=cb2faddd68a645e09fc227a8238be6a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:05.448433 1245 tablet_service.cc:1467] Processing CreateTablet for tablet 98a2caaab97d477a98bec54a442452d6 (DEFAULT_TABLE table=test-table [id=cb2faddd68a645e09fc227a8238be6a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:05.448531 927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98a2caaab97d477a98bec54a442452d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:05.448726 1245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98a2caaab97d477a98bec54a442452d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:05.448803 1087 tablet_service.cc:1467] Processing CreateTablet for tablet 98a2caaab97d477a98bec54a442452d6 (DEFAULT_TABLE table=test-table [id=cb2faddd68a645e09fc227a8238be6a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:05.449079 1087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98a2caaab97d477a98bec54a442452d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:05.451227 1350 tablet_bootstrap.cc:492] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: Bootstrap starting.
I20260811 22:26:05.451571 1347 tablet_bootstrap.cc:492] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: Bootstrap starting.
I20260811 22:26:05.452270 1350 tablet_bootstrap.cc:654] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:05.452510 1347 tablet_bootstrap.cc:654] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:05.452728 765 catalog_manager.cc:4841] TS 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637): tablet 737ee195226b48609bcb05eafb814384 (table test-table [id=4c843e558ee84ca99578038cd77725fc]) successfully deleted
I20260811 22:26:05.453454 1349 tablet_bootstrap.cc:492] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb: Bootstrap starting.
I20260811 22:26:05.454428 1349 tablet_bootstrap.cc:654] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:05.456282 1349 tablet_bootstrap.cc:492] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb: No bootstrap required, opened a new log
I20260811 22:26:05.456482 1349 ts_tablet_manager.cc:1397] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:05.457576 1349 raft_consensus.cc:348] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.457840 1349 raft_consensus.cc:374] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:05.457990 1349 raft_consensus.cc:729] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e38521232d4cccb66debda5a10f7bb, State: Initialized, Role: FOLLOWER
I20260811 22:26:05.458455 1349 consensus_queue.cc:260] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.461776 1349 ts_tablet_manager.cc:1428] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:26:05.463022 1347 tablet_bootstrap.cc:492] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: No bootstrap required, opened a new log
I20260811 22:26:05.463135 1347 ts_tablet_manager.cc:1397] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260811 22:26:05.464100 1347 raft_consensus.cc:348] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.464243 1347 raft_consensus.cc:374] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:05.464284 1347 raft_consensus.cc:729] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08991788adaa4e9cabe6706a70e3b6f3, State: Initialized, Role: FOLLOWER
I20260811 22:26:05.464439 1347 consensus_queue.cc:260] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.464741 1347 ts_tablet_manager.cc:1428] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:26:05.472987 1350 tablet_bootstrap.cc:492] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: No bootstrap required, opened a new log
I20260811 22:26:05.473150 1350 ts_tablet_manager.cc:1397] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20260811 22:26:05.473625 1350 raft_consensus.cc:348] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.473778 1350 raft_consensus.cc:374] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:05.473817 1350 raft_consensus.cc:729] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e5d5a5395349f6adc7ed239db856c7, State: Initialized, Role: FOLLOWER
I20260811 22:26:05.473969 1350 consensus_queue.cc:260] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.474308 1350 ts_tablet_manager.cc:1428] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:26:05.659169 1435 raft_consensus.cc:482] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:26:05.659343 1435 raft_consensus.cc:504] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.659767 1435 leader_election.cc:288] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637), c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369)
I20260811 22:26:05.660413 950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2caaab97d477a98bec54a442452d6" candidate_uuid: "08991788adaa4e9cabe6706a70e3b6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82e38521232d4cccb66debda5a10f7bb" is_pre_election: true
I20260811 22:26:05.660419 1266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2caaab97d477a98bec54a442452d6" candidate_uuid: "08991788adaa4e9cabe6706a70e3b6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5e5d5a5395349f6adc7ed239db856c7" is_pre_election: true
I20260811 22:26:05.660609 950 raft_consensus.cc:2454] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08991788adaa4e9cabe6706a70e3b6f3 in term 0.
I20260811 22:26:05.660616 1266 raft_consensus.cc:2454] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08991788adaa4e9cabe6706a70e3b6f3 in term 0.
I20260811 22:26:05.661068 1034 leader_election.cc:302] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08991788adaa4e9cabe6706a70e3b6f3, 82e38521232d4cccb66debda5a10f7bb, c5e5d5a5395349f6adc7ed239db856c7; no voters:
I20260811 22:26:05.661268 1435 raft_consensus.cc:2781] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:26:05.661343 1435 raft_consensus.cc:482] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:26:05.661376 1435 raft_consensus.cc:3037] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:05.662477 1435 raft_consensus.cc:504] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.662825 1435 leader_election.cc:288] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637), c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369)
I20260811 22:26:05.663264 1266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2caaab97d477a98bec54a442452d6" candidate_uuid: "08991788adaa4e9cabe6706a70e3b6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5e5d5a5395349f6adc7ed239db856c7"
I20260811 22:26:05.663264 950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2caaab97d477a98bec54a442452d6" candidate_uuid: "08991788adaa4e9cabe6706a70e3b6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82e38521232d4cccb66debda5a10f7bb"
I20260811 22:26:05.663415 950 raft_consensus.cc:3037] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:05.663415 1266 raft_consensus.cc:3037] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:05.664779 950 raft_consensus.cc:2454] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08991788adaa4e9cabe6706a70e3b6f3 in term 1.
I20260811 22:26:05.664779 1266 raft_consensus.cc:2454] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08991788adaa4e9cabe6706a70e3b6f3 in term 1.
I20260811 22:26:05.665153 1033 leader_election.cc:302] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08991788adaa4e9cabe6706a70e3b6f3, 82e38521232d4cccb66debda5a10f7bb; no voters:
I20260811 22:26:05.665441 1435 raft_consensus.cc:2781] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:05.665558 1435 raft_consensus.cc:686] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 08991788adaa4e9cabe6706a70e3b6f3, State: Running, Role: LEADER
I20260811 22:26:05.665714 1435 consensus_queue.cc:237] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } }
I20260811 22:26:05.667665 780 catalog_manager.cc:5495] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194). New cstate: current_term: 1 leader_uuid: "08991788adaa4e9cabe6706a70e3b6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08991788adaa4e9cabe6706a70e3b6f3" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 44717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 } health_report { overall_health: UNKNOWN } } }
I20260811 22:26:05.720556 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.716538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:37369 (local address 127.24.71.194:35673)
0811 22:26:05.717031 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:05.717045 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:05.717192 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:05.717568 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:05.717570 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:05.717579 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:26:05.717700 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.717708 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.718483 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.718487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.719206 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.719216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.719693 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.719697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.719795 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.719871 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.720216 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.720225 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:26:05.720316 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:26:05.720414 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":156,"threads_started":1}
I20260811 22:26:05.720554 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.716723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:35673 (local address 127.24.71.195:37369)
0811 22:26:05.716919 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.716925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.717132 (+ 207us) server_negotiation.cc:407] Connection header received
0811 22:26:05.717337 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.717340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.717453 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.717551 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:05.717826 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.718351 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.719332 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.719567 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.719684 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.720046 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.720126 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.720211 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.720330 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.720395 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 22:26:05.727466 1266 raft_consensus.cc:1261] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Refusing update from remote peer 08991788adaa4e9cabe6706a70e3b6f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:05.727545 950 raft_consensus.cc:1261] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Refusing update from remote peer 08991788adaa4e9cabe6706a70e3b6f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:05.727936 1435 consensus_queue.cc:1035] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5e5d5a5395349f6adc7ed239db856c7" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 37369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:05.728174 1428 consensus_queue.cc:1035] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82e38521232d4cccb66debda5a10f7bb" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:05.746898 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.743155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:51161 (local address 127.24.71.194:44717)
0811 22:26:05.743307 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:05.743312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:05.743475 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:26:05.743586 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:05.743589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:05.743634 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:05.743713 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:05.744040 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.744648 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.745633 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.745876 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.745976 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.746330 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.746421 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.746584 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:05.746699 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 22:26:05.746754 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:26:05.746908 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.743048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:44717 (local address 127.24.71.195:51161)
0811 22:26:05.743345 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:05.743387 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:05.743501 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:05.743750 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:05.743754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:05.743764 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:26:05.743906 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.743913 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.744778 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.744781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.745484 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:05.745496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:05.746004 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:05.746009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:05.746134 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:05.746171 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:26:05.746586 (+ 415us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:26:05.746595 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:26:05.746681 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:26:05.746777 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":129,"threads_started":1}
I20260811 22:26:05.768152 780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:05.768316 780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:05.770491 780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7fd5b7a87874f8c9a94fae1a1dd3564: Sending DeleteTablet for 3 replicas of tablet 98a2caaab97d477a98bec54a442452d6
I20260811 22:26:05.771242 927 tablet_service.cc:1514] Processing DeleteTablet for tablet 98a2caaab97d477a98bec54a442452d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:05 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:39642
I20260811 22:26:05.771348 1087 tablet_service.cc:1514] Processing DeleteTablet for tablet 98a2caaab97d477a98bec54a442452d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:05 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60152
I20260811 22:26:05.771512 1372 tablet_replica.cc:330] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb: stopping tablet replica
I20260811 22:26:05.771587 1373 tablet_replica.cc:330] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: stopping tablet replica
I20260811 22:26:05.771700 1372 raft_consensus.cc:2229] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:05.771764 1373 raft_consensus.cc:2229] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:26:05.771907 1372 raft_consensus.cc:2258] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:05.772032 1373 raft_consensus.cc:2258] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:05.772317 1245 tablet_service.cc:1514] Processing DeleteTablet for tablet 98a2caaab97d477a98bec54a442452d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:05 UTC) from {username='oryx', principal='oryx/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58180
I20260811 22:26:05.772531 1374 tablet_replica.cc:330] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: stopping tablet replica
I20260811 22:26:05.772675 1374 raft_consensus.cc:2229] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:05.772825 1374 raft_consensus.cc:2258] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:05.773056 1372 ts_tablet_manager.cc:1905] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:05.773209 1373 ts_tablet_manager.cc:1905] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:05.773680 1374 ts_tablet_manager.cc:1905] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:05.775755 1372 ts_tablet_manager.cc:1918] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:26:05.775874 1372 log.cc:1196] T 98a2caaab97d477a98bec54a442452d6 P 82e38521232d4cccb66debda5a10f7bb: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/98a2caaab97d477a98bec54a442452d6
I20260811 22:26:05.775820 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 858
I20260811 22:26:05.779965 1374 ts_tablet_manager.cc:1918] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:26:05.780058 1374 log.cc:1196] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/98a2caaab97d477a98bec54a442452d6
I20260811 22:26:05.780105 1373 ts_tablet_manager.cc:1918] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:26:05.780189 1373 log.cc:1196] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/98a2caaab97d477a98bec54a442452d6
I20260811 22:26:05.780364 1374 ts_tablet_manager.cc:1939] T 98a2caaab97d477a98bec54a442452d6 P c5e5d5a5395349f6adc7ed239db856c7: Deleting consensus metadata
I20260811 22:26:05.780504 1373 ts_tablet_manager.cc:1939] T 98a2caaab97d477a98bec54a442452d6 P 08991788adaa4e9cabe6706a70e3b6f3: Deleting consensus metadata
I20260811 22:26:05.781224 766 catalog_manager.cc:4841] TS c5e5d5a5395349f6adc7ed239db856c7 (127.24.71.195:37369): tablet 98a2caaab97d477a98bec54a442452d6 (table test-table [id=cb2faddd68a645e09fc227a8238be6a5]) successfully deleted
I20260811 22:26:05.781780 767 catalog_manager.cc:4841] TS 08991788adaa4e9cabe6706a70e3b6f3 (127.24.71.194:44717): tablet 98a2caaab97d477a98bec54a442452d6 (table test-table [id=cb2faddd68a645e09fc227a8238be6a5]) successfully deleted
W20260811 22:26:05.789424 765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:40637 (error 108)
I20260811 22:26:05.789533 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 1012
I20260811 22:26:05.793680 1454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:05.793056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40637 (local address 127.0.0.1:39660)
0811 22:26:05.793450 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:05.793540 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:40637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":226,"threads_started":1}
W20260811 22:26:05.794112 765 catalog_manager.cc:4584] TS 82e38521232d4cccb66debda5a10f7bb (127.24.71.193:40637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 98a2caaab97d477a98bec54a442452d6: Network error: Client connection negotiation failed: client connection to 127.24.71.193:40637: connect: Connection refused (error 111)
I20260811 22:26:05.800928 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 1169
I20260811 22:26:05.811098 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 746
2026-08-11T22:26:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3725 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:26:05 dist-test-slave-mvvj krb5kdc[1457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:26:05 dist-test-slave-mvvj krb5kdc[1457](info): set up 2 sockets
Aug 11 22:26:05 dist-test-slave-mvvj krb5kdc[1457](info): commencing operation
krb5kdc: starting...
W20260811 22:26:06.944000 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:26:07 dist-test-slave-mvvj krb5kdc[1457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:26:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:26:07.082983 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40215
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:46245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:40215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:07.204054 1472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:07.204409 1472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:07.204502 1472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:07.208683 1472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:07.208777 1472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:07.208815 1472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:07.208875 1472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:07.208912 1472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:07.214490 1472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:46245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:40215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:07.215715 1472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:07.216868 1472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:07.226055 1480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:07.226033 1477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:07.226033 1478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:07.227015 1472 server_base.cc:1034] running on GCE node
I20260811 22:26:07.227717 1472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:07.228977 1472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:07.230182 1472 hybrid_clock.cc:648] HybridClock initialized: now 1786487167230153 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T22:26:07Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1425 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:26:07 dist-test-slave-mvvj krb5kdc[1485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:26:07 dist-test-slave-mvvj krb5kdc[1485](info): set up 2 sockets
Aug 11 22:26:07 dist-test-slave-mvvj krb5kdc[1485](info): commencing operation
krb5kdc: starting...
W20260811 22:26:08.347726 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:26:08 dist-test-slave-mvvj krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:26:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:26:08.487558 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:36577
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:36577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:08.606372 1501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:08.606724 1501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:08.606815 1501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:08.611012 1501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:08.611121 1501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:08.611163 1501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:08.611208 1501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:08.611259 1501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:08.616843 1501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:36577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:36577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:08.618117 1501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:08.619243 1501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:08.629287 1510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:08.629405 1508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:08.629287 1507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:08.629561 1501 server_base.cc:1034] running on GCE node
I20260811 22:26:08.630265 1501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:08.631494 1501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:08.632710 1501 hybrid_clock.cc:648] HybridClock initialized: now 1786487168632690 us; error 45 us; skew 500 ppm
Aug 11 22:26:08 dist-test-slave-mvvj krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487168, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:08.642131 1501 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:26:08.643371 1501 webserver.cc:460] Webserver started at http://127.24.71.254:40329/ using document root <none> and password file <none>
I20260811 22:26:08.644100 1501 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:08.644203 1501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:08.644464 1501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:08.646770 1501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "40e30b451ea64d94821e779248e1cd9f"
format_stamp: "Formatted at 2026-08-11 22:26:08 on dist-test-slave-mvvj"
server_key: "35dab222c0804fc857bc651525287bca"
server_key_iv: "4ca09a99e6bace0b86b78eeb5449b520"
server_key_version: "encryptionkey@0"
I20260811 22:26:08.647696 1501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "40e30b451ea64d94821e779248e1cd9f"
format_stamp: "Formatted at 2026-08-11 22:26:08 on dist-test-slave-mvvj"
server_key: "35dab222c0804fc857bc651525287bca"
server_key_iv: "4ca09a99e6bace0b86b78eeb5449b520"
server_key_version: "encryptionkey@0"
I20260811 22:26:08.652158 1501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:26:08.654963 1517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:08.656342 1501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:08.656510 1501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "40e30b451ea64d94821e779248e1cd9f"
format_stamp: "Formatted at 2026-08-11 22:26:08 on dist-test-slave-mvvj"
server_key: "35dab222c0804fc857bc651525287bca"
server_key_iv: "4ca09a99e6bace0b86b78eeb5449b520"
server_key_version: "encryptionkey@0"
I20260811 22:26:08.656648 1501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:08.684211 1501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:08.687994 1501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:08.688274 1501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:08.697034 1501 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:36577
I20260811 22:26:08.697070 1575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:36577 every 8 connection(s)
I20260811 22:26:08.698149 1501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:26:08.701385 1576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:08.706326 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 1501
I20260811 22:26:08.706444 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:26:08.706764 24863 external_mini_cluster.cc:1428] Setting key 1ff09808eaaa65e27d964f3f0f0251e0
I20260811 22:26:08.708021 1576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: Bootstrap starting.
I20260811 22:26:08.710994 1576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:08.711964 1576 log.cc:824] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:08.714289 1576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: No bootstrap required, opened a new log
Aug 11 22:26:08 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:26:08.719496 1576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } }
I20260811 22:26:08.719875 1576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:08.719940 1576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e30b451ea64d94821e779248e1cd9f, State: Initialized, Role: FOLLOWER
I20260811 22:26:08.720479 1576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } }
I20260811 22:26:08.720676 1576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:26:08.720767 1576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:26:08.720875 1576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:08.722234 1576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } }
I20260811 22:26:08.722640 1576 leader_election.cc:302] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40e30b451ea64d94821e779248e1cd9f; no voters:
I20260811 22:26:08.723011 1576 leader_election.cc:288] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:26:08.723181 1581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:08.723453 1581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 1 LEADER]: Becoming Leader. State: Replica: 40e30b451ea64d94821e779248e1cd9f, State: Running, Role: LEADER
I20260811 22:26:08.723788 1579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:08.708374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34984 (local address 127.24.71.254:36577)
0811 22:26:08.708878 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:08.708912 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:08.708955 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:26:08.709894 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:08.709919 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:08.710357 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:08.710747 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:08.711576 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.712981 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.713880 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:08.714579 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:08.714757 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:08.717678 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:08.717689 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:08.717719 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:08.717766 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:08.720046 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:08.720658 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:08.720664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:08.720671 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:08.720839 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:08.721185 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:08.721190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:08.721192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:08.721774 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:08.721997 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:08.722327 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 22:26:08.722686 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":171,"threads_started":1}
I20260811 22:26:08.723919 1581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } }
I20260811 22:26:08.724473 1576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:08.725924 1583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40e30b451ea64d94821e779248e1cd9f. Latest consensus state: current_term: 1 leader_uuid: "40e30b451ea64d94821e779248e1cd9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } } }
I20260811 22:26:08.726147 1583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:08.726651 1592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:08.726924 1582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40e30b451ea64d94821e779248e1cd9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } } }
I20260811 22:26:08.727694 1582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:08.730158 1592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:08.735862 1592 catalog_manager.cc:1347] Generated new cluster ID: f66ebe1b033a44b180a2e929db709bc5
I20260811 22:26:08.735948 1592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:08.749713 1592 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:26:08.750797 1592 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:08.762199 1592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: Generated new TSK 0
I20260811 22:26:08.762977 1592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:26:08.804155 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:36577
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:08.927320 1603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:08.927628 1603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:08.927675 1603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:08.931934 1603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:08.932053 1603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:08.932117 1603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:26:08.937696 1603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:36577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:08.938916 1603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:08.940029 1603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:08.950440 1608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:08.950512 1611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:08.950625 1609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:08.950714 1603 server_base.cc:1034] running on GCE node
I20260811 22:26:08.951362 1603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:08.952188 1603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:08.953385 1603 hybrid_clock.cc:648] HybridClock initialized: now 1786487168953366 us; error 32 us; skew 500 ppm
Aug 11 22:26:08 dist-test-slave-mvvj krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487168, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:08.956700 1603 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:26:08.957751 1603 webserver.cc:460] Webserver started at http://127.24.71.193:45877/ using document root <none> and password file <none>
I20260811 22:26:08.958376 1603 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:08.958477 1603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:08.958734 1603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:08.961073 1603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "393501db4c2549f89abd93a04561c75b"
format_stamp: "Formatted at 2026-08-11 22:26:08 on dist-test-slave-mvvj"
server_key: "94babb344fd96b94e9ea2572f7985bf5"
server_key_iv: "df9ab3933755106d8c3e4dc0b965a114"
server_key_version: "encryptionkey@0"
I20260811 22:26:08.961757 1603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "393501db4c2549f89abd93a04561c75b"
format_stamp: "Formatted at 2026-08-11 22:26:08 on dist-test-slave-mvvj"
server_key: "94babb344fd96b94e9ea2572f7985bf5"
server_key_iv: "df9ab3933755106d8c3e4dc0b965a114"
server_key_version: "encryptionkey@0"
I20260811 22:26:08.966032 1603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:26:08.968734 1620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:08.970054 1603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:26:08.970209 1603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "393501db4c2549f89abd93a04561c75b"
format_stamp: "Formatted at 2026-08-11 22:26:08 on dist-test-slave-mvvj"
server_key: "94babb344fd96b94e9ea2572f7985bf5"
server_key_iv: "df9ab3933755106d8c3e4dc0b965a114"
server_key_version: "encryptionkey@0"
I20260811 22:26:08.970340 1603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:08.983551 1603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:08.987023 1603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:08.987250 1603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:08.987972 1603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:08.988909 1603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:08.988987 1603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:08.989064 1603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:08.989110 1603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:08.999293 1603 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:38345
I20260811 22:26:08.999313 1744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:38345 every 8 connection(s)
I20260811 22:26:09.000370 1603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:26:09.002434 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 1603
I20260811 22:26:09.002568 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:26:09.002910 24863 external_mini_cluster.cc:1428] Setting key be90911e65f341bec3c00f58ddb271df
Aug 11 22:26:09 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487168, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:26:09.017400 1579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:09.002683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:35489 (local address 127.24.71.254:36577)
0811 22:26:09.002835 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:09.002845 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:09.003880 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 22:26:09.005586 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:09.005590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:09.005646 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:09.005752 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:09.007240 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.007774 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.009698 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.009936 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.010110 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:09.013305 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:09.013308 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:09.013310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:09.013337 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:09.015029 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:09.015597 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:09.015601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:09.015603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:09.015725 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:09.016149 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:09.016153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:09.016155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:09.016366 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:09.016483 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:09.017067 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 22:26:09.017216 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
I20260811 22:26:09.018527 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:09.003112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:35489)
0811 22:26:09.003706 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:09.003753 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:09.005353 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:09.005940 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:09.005948 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:09.006327 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:09.007056 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:09.007080 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.007982 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.007988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:09.009526 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:09.009541 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.010169 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.010176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:09.010438 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:09.011139 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:09.011169 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:09.013088 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:09.015169 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:09.015176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:09.015194 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:09.015483 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:09.015857 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:09.015864 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:09.015866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:09.016035 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:09.016483 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:09.016492 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:09.016778 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 22:26:09.017005 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 22:26:09.017467 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":161,"threads_started":1}
I20260811 22:26:09.020049 1745 heartbeater.cc:347] Connected to a master server at 127.24.71.254:36577
I20260811 22:26:09.020349 1745 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:09.020953 1745 heartbeater.cc:511] Master 127.24.71.254:36577 requested a full tablet report, sending...
I20260811 22:26:09.022650 1534 ts_manager.cc:194] Registered new tserver with Master: 393501db4c2549f89abd93a04561c75b (127.24.71.193:38345)
I20260811 22:26:09.024029 1534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:35489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:26:09.039019 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:36577
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:09.157804 1750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:09.158133 1750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:09.158215 1750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:09.162492 1750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:09.162563 1750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:09.162648 1750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:26:09.168107 1750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:36577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:09.169209 1750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:09.170248 1750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:09.180394 1755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:09.180394 1757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:09.180500 1750 server_base.cc:1034] running on GCE node
W20260811 22:26:09.180512 1759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:09.181093 1750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:09.181800 1750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:09.182997 1750 hybrid_clock.cc:648] HybridClock initialized: now 1786487169182983 us; error 30 us; skew 500 ppm
Aug 11 22:26:09 dist-test-slave-mvvj krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487169, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:09.185926 1750 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:26:09.186869 1750 webserver.cc:460] Webserver started at http://127.24.71.194:44809/ using document root <none> and password file <none>
I20260811 22:26:09.187407 1750 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:09.187467 1750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:09.187639 1750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:09.189805 1750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "0ea8808d91574bb6ace82dda4f44e5e0"
format_stamp: "Formatted at 2026-08-11 22:26:09 on dist-test-slave-mvvj"
server_key: "03e1127691a710f35718897734f98fc9"
server_key_iv: "8713c2dc8f7e01fa51011a75fe4f5fe7"
server_key_version: "encryptionkey@0"
I20260811 22:26:09.190354 1750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "0ea8808d91574bb6ace82dda4f44e5e0"
format_stamp: "Formatted at 2026-08-11 22:26:09 on dist-test-slave-mvvj"
server_key: "03e1127691a710f35718897734f98fc9"
server_key_iv: "8713c2dc8f7e01fa51011a75fe4f5fe7"
server_key_version: "encryptionkey@0"
I20260811 22:26:09.194235 1750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:26:09.197001 1766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:09.198267 1750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:09.198395 1750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "0ea8808d91574bb6ace82dda4f44e5e0"
format_stamp: "Formatted at 2026-08-11 22:26:09 on dist-test-slave-mvvj"
server_key: "03e1127691a710f35718897734f98fc9"
server_key_iv: "8713c2dc8f7e01fa51011a75fe4f5fe7"
server_key_version: "encryptionkey@0"
I20260811 22:26:09.198488 1750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:09.235556 1750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:09.239270 1750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:09.239466 1750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:09.240236 1750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:09.241132 1750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:09.241181 1750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:09.241225 1750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:09.241240 1750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:09.250999 1750 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:41687
I20260811 22:26:09.251022 1892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:41687 every 8 connection(s)
I20260811 22:26:09.252136 1750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:26:09.257535 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 1750
I20260811 22:26:09.257642 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:26:09.257959 24863 external_mini_cluster.cc:1428] Setting key 29cb385cbb8d3ad97d32a35d1ed3a5e3
Aug 11 22:26:09 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487169, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:26:09.267573 1579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:09.254168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:34061 (local address 127.24.71.254:36577)
0811 22:26:09.254330 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:09.254335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:09.255240 (+ 905us) server_negotiation.cc:407] Connection header received
0811 22:26:09.256675 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:09.256678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:09.256728 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:09.256818 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:09.258111 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.258754 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.260183 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.260608 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.260735 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:09.263891 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:09.263896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:09.263900 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:09.263939 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:09.265421 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:09.265996 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:09.266000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:09.266001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:09.266078 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:09.266441 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:09.266444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:09.266446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:09.266638 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:09.266752 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:09.267267 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 22:26:09.267398 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:26:09.268618 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:09.254614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:34061)
0811 22:26:09.255080 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:09.255125 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:09.256450 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:09.256995 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:09.257002 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:09.257366 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:09.257928 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:09.257943 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.258912 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.258915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:09.260052 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:09.260063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.260811 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.260819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:09.261024 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:09.261746 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:09.261773 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:09.263660 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:09.265552 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:09.265558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:09.265575 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:09.265869 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:09.266177 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:09.266180 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:09.266182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:09.266330 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:09.266752 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:09.266760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:09.267022 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 22:26:09.267222 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 22:26:09.267576 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":120,"threads_started":1}
I20260811 22:26:09.269942 1893 heartbeater.cc:347] Connected to a master server at 127.24.71.254:36577
I20260811 22:26:09.270177 1893 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:09.270789 1893 heartbeater.cc:511] Master 127.24.71.254:36577 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:26:09.272114 1534 ts_manager.cc:194] Registered new tserver with Master: 0ea8808d91574bb6ace82dda4f44e5e0 (127.24.71.194:41687)
I20260811 22:26:09.272725 1534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:34061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:26:09.291541 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:36577
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:09.407564 1898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:09.407925 1898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:09.408030 1898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:09.412119 1898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:09.412225 1898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:09.412322 1898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:26:09.417793 1898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:36577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:09.419019 1898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:09.420234 1898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:09.430670 1905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:09.430760 1898 server_base.cc:1034] running on GCE node
W20260811 22:26:09.430663 1907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:09.430670 1904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:09.431435 1898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:09.432163 1898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:09.433329 1898 hybrid_clock.cc:648] HybridClock initialized: now 1786487169433295 us; error 45 us; skew 500 ppm
Aug 11 22:26:09 dist-test-slave-mvvj krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487169, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:09.436380 1898 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:26:09.437419 1898 webserver.cc:460] Webserver started at http://127.24.71.195:40413/ using document root <none> and password file <none>
I20260811 22:26:09.438027 1898 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:09.438093 1898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:09.438339 1898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:09.440682 1898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "539e8e44e6e04514a77e687f72e5f50c"
format_stamp: "Formatted at 2026-08-11 22:26:09 on dist-test-slave-mvvj"
server_key: "5945732f693d71107ce8bdf64d742ab9"
server_key_iv: "2087f0e0c0e4049d05553ef075927076"
server_key_version: "encryptionkey@0"
I20260811 22:26:09.441380 1898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "539e8e44e6e04514a77e687f72e5f50c"
format_stamp: "Formatted at 2026-08-11 22:26:09 on dist-test-slave-mvvj"
server_key: "5945732f693d71107ce8bdf64d742ab9"
server_key_iv: "2087f0e0c0e4049d05553ef075927076"
server_key_version: "encryptionkey@0"
I20260811 22:26:09.445504 1898 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:26:09.448437 1914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:09.449893 1898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:09.450100 1898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "539e8e44e6e04514a77e687f72e5f50c"
format_stamp: "Formatted at 2026-08-11 22:26:09 on dist-test-slave-mvvj"
server_key: "5945732f693d71107ce8bdf64d742ab9"
server_key_iv: "2087f0e0c0e4049d05553ef075927076"
server_key_version: "encryptionkey@0"
I20260811 22:26:09.450248 1898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:09.470182 1898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:09.473757 1898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:09.473984 1898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:09.474713 1898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:09.475626 1898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:09.475708 1898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:09.475790 1898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:09.475871 1898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 22:26:09.485683 1898 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:38951
I20260811 22:26:09.485708 2032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:38951 every 8 connection(s)
I20260811 22:26:09.486645 1898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:26:09.489648 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 1898
I20260811 22:26:09.489809 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:26:09.490203 24863 external_mini_cluster.cc:1428] Setting key 736f590543175b3a56c297dc675e0093
Aug 11 22:26:09 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487169, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:26:09.502082 1579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:09.488757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:50315 (local address 127.24.71.254:36577)
0811 22:26:09.488966 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:09.488970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:09.489911 (+ 941us) server_negotiation.cc:407] Connection header received
0811 22:26:09.491801 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:09.491806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:09.491901 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:09.491997 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:09.493355 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.494013 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.495169 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.495405 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.495505 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:09.498281 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:09.498285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:09.498287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:09.498327 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:09.499746 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:09.500366 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:09.500370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:09.500373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:09.500472 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:09.500855 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:09.500860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:09.500862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:09.501073 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:09.501195 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:09.501749 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 22:26:09.501891 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 22:26:09.503113 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:09.489138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:50315)
0811 22:26:09.489725 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:09.489796 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:09.491512 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:09.492166 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:09.492173 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:09.492688 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:09.493197 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:09.493211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.494143 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.494146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:09.495043 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:09.495053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:09.495551 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:09.495554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:09.495707 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:09.496428 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:09.496446 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:09.498086 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:09.499917 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:09.499924 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:09.499941 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:09.500245 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:09.500581 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:09.500584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:09.500587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:09.500744 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:09.501192 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:09.501200 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:09.501480 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 22:26:09.501688 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 22:26:09.502117 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":129,"threads_started":1}
I20260811 22:26:09.504396 2033 heartbeater.cc:347] Connected to a master server at 127.24.71.254:36577
I20260811 22:26:09.504654 2033 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:09.505191 2033 heartbeater.cc:511] Master 127.24.71.254:36577 requested a full tablet report, sending...
I20260811 22:26:09.506278 1534 ts_manager.cc:194] Registered new tserver with Master: 539e8e44e6e04514a77e687f72e5f50c (127.24.71.195:38951)
I20260811 22:26:09.506945 1534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:50315
I20260811 22:26:09.516852 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab.
Entry for principal oryx/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab.
WARNING: no policy specified for oryx/127.24.71.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab.
Entry for principal oryx/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab.
WARNING: no policy specified for oryx/127.24.71.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.71.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab.
Entry for principal oryx/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab.
I20260811 22:26:09.603156 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 1603
I20260811 22:26:09.611155 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 1750
I20260811 22:26:09.619637 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 1898
I20260811 22:26:09.627594 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 1501
I20260811 22:26:09.637401 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:36577
--webserver_interface=127.24.71.254
--webserver_port=40329
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:36577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:09.765600 2044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:09.765959 2044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:09.766055 2044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:09.770668 2044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:09.770745 2044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:09.770766 2044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:09.770783 2044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:09.770798 2044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:09.776652 2044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:36577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:36577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=40329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:09.777884 2044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:09.779078 2044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:09.788849 2050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:09.788864 2051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:09.788920 2044 server_base.cc:1034] running on GCE node
W20260811 22:26:09.788849 2053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:09.789928 2044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:09.791231 2044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:09.792409 2044 hybrid_clock.cc:648] HybridClock initialized: now 1786487169792390 us; error 37 us; skew 500 ppm
Aug 11 22:26:09 dist-test-slave-mvvj krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487169, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:09.795539 2044 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:26:09.796670 2044 webserver.cc:460] Webserver started at http://127.24.71.254:40329/ using document root <none> and password file <none>
I20260811 22:26:09.797338 2044 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:09.797466 2044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:09.802485 2044 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 22:26:09.805311 2060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:09.806753 2044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:26:09.806953 2044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "40e30b451ea64d94821e779248e1cd9f"
format_stamp: "Formatted at 2026-08-11 22:26:08 on dist-test-slave-mvvj"
server_key: "35dab222c0804fc857bc651525287bca"
server_key_iv: "4ca09a99e6bace0b86b78eeb5449b520"
server_key_version: "encryptionkey@0"
I20260811 22:26:09.807660 2044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:09.823664 2044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:09.827138 2044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:09.827381 2044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:09.835420 2044 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:36577
I20260811 22:26:09.835610 2114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:36577 every 8 connection(s)
I20260811 22:26:09.836699 2044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:26:09.841722 2115 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:26:09.843710 2115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: Bootstrap starting.
I20260811 22:26:09.845440 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 2044
I20260811 22:26:09.846012 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:38345
--local_ip_for_outbound_sockets=127.24.71.193
--tserver_master_addrs=127.24.71.254:36577
--webserver_port=45877
--webserver_interface=127.24.71.193
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.71.193
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:26:09.864461 2115 log.cc:824] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:09.871613 2115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:26:09.872068 2115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: Bootstrap complete.
I20260811 22:26:09.876652 2115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } }
I20260811 22:26:09.877072 2115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e30b451ea64d94821e779248e1cd9f, State: Initialized, Role: FOLLOWER
I20260811 22:26:09.877607 2115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } }
I20260811 22:26:09.877730 2115 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:26:09.877830 2115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:26:09.877954 2115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:26:09.880080 2115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } }
I20260811 22:26:09.880430 2115 leader_election.cc:302] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40e30b451ea64d94821e779248e1cd9f; no voters:
I20260811 22:26:09.880805 2115 leader_election.cc:288] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:26:09.880890 2119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:26:09.881081 2119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [term 2 LEADER]: Becoming Leader. State: Replica: 40e30b451ea64d94821e779248e1cd9f, State: Running, Role: LEADER
I20260811 22:26:09.881484 2119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } }
I20260811 22:26:09.882130 2115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:09.882893 2120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "40e30b451ea64d94821e779248e1cd9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } } }
I20260811 22:26:09.883037 2120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:09.883419 2121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40e30b451ea64d94821e779248e1cd9f. Latest consensus state: current_term: 2 leader_uuid: "40e30b451ea64d94821e779248e1cd9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e30b451ea64d94821e779248e1cd9f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 36577 } } }
I20260811 22:26:09.883529 2121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:09.883574 2126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:09.887144 2126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:09.887872 2126 catalog_manager.cc:1259] Loaded cluster ID: f66ebe1b033a44b180a2e929db709bc5
I20260811 22:26:09.887938 2126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:09.890363 2126 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:09.891142 2126 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 40e30b451ea64d94821e779248e1cd9f: Loaded TSK: 0
I20260811 22:26:09.892592 2126 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:26:09.970054 2117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:09.970377 2117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:09.970458 2117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:09.974848 2117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:09.974952 2117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:09.975090 2117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:26:09.980800 2117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.193.keytab
--principal=oryx/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:38345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=45877
--tserver_master_addrs=127.24.71.254:36577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:09.982000 2117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:09.983522 2117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:09.993710 2143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:09.993710 2147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:09.993695 2144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:09.994001 2117 server_base.cc:1034] running on GCE node
I20260811 22:26:09.994606 2117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:09.995311 2117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:09.996505 2117 hybrid_clock.cc:648] HybridClock initialized: now 1786487169996474 us; error 49 us; skew 500 ppm
Aug 11 22:26:09 dist-test-slave-mvvj krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487169, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:09.999711 2117 init.cc:377] Logged in from keytab as oryx/127.24.71.193@KRBTEST.COM (short username oryx)
I20260811 22:26:10.000828 2117 webserver.cc:460] Webserver started at http://127.24.71.193:45877/ using document root <none> and password file <none>
I20260811 22:26:10.001474 2117 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:10.001569 2117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:10.006119 2117 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:26:10.008620 2155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:10.010116 2117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:10.010280 2117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "393501db4c2549f89abd93a04561c75b"
format_stamp: "Formatted at 2026-08-11 22:26:08 on dist-test-slave-mvvj"
server_key: "94babb344fd96b94e9ea2572f7985bf5"
server_key_iv: "df9ab3933755106d8c3e4dc0b965a114"
server_key_version: "encryptionkey@0"
I20260811 22:26:10.010911 2117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:10.032454 2117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:10.036320 2117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:10.036577 2117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:10.037256 2117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:10.038172 2117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:10.038247 2117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:10.038337 2117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:10.038391 2117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:10.048462 2117 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:38345
I20260811 22:26:10.048491 2274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:38345 every 8 connection(s)
I20260811 22:26:10.049542 2117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:26:10.054626 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 2117
I20260811 22:26:10.055305 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:41687
--local_ip_for_outbound_sockets=127.24.71.194
--tserver_master_addrs=127.24.71.254:36577
--webserver_port=44809
--webserver_interface=127.24.71.194
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.71.194
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:26:10.066066 2277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:10.052066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:60891 (local address 127.24.71.254:36577)
0811 22:26:10.052569 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.052592 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.053073 (+ 481us) server_negotiation.cc:407] Connection header received
0811 22:26:10.054974 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.054994 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.055589 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.056058 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.057248 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.058287 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.059548 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.060148 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.060343 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.065154 (+ 4811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:60891: BlockingRecv error: recv got EOF from 127.24.71.193:60891 (error 108)
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":150,"threads_started":1}
I20260811 22:26:10.066195 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.052204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:60891)
0811 22:26:10.052740 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:10.052814 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:10.054302 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:10.056134 (+ 1832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:10.056142 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:10.056540 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:10.057044 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.057061 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.058477 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.058485 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.059394 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.059405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.060338 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.060342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.060547 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.061269 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:10.061287 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:10.064886 (+ 3599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":162,"threads_started":1}
W20260811 22:26:10.066936 2275 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:36577 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.71.254:36577: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:10.073580 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.067563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:56909)
0811 22:26:10.067701 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:10.067720 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:10.067821 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:10.068209 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:10.068213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:10.068410 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:10.068520 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.068526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.069345 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.069352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.069974 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.069983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.070495 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.070499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.070615 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.071205 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:10.071218 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:10.073431 (+ 2213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:26:10.073761 2277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:10.067681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:56909 (local address 127.24.71.254:36577)
0811 22:26:10.067825 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.067855 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.067872 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:26:10.068023 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.068038 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.068091 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.068192 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.068650 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.069227 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.070109 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.070359 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.070492 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.073639 (+ 3147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:56909: BlockingRecv error: recv got EOF from 127.24.71.193:56909 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:10.080189 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.074343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:52007)
0811 22:26:10.074510 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:10.074527 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:10.074614 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:10.074894 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:10.074897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:10.075076 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:10.075175 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.075181 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.076007 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.076014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.076936 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.076949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.077393 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.077397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.077499 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.077970 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:10.077984 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:10.080041 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 22:26:10.080262 2277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:10.074486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:52007 (local address 127.24.71.254:36577)
0811 22:26:10.074623 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.074627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.074641 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:10.074699 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.074701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.074740 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.074810 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.075324 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.075816 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.077070 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.077282 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.077366 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.080155 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:52007: BlockingRecv error: recv got EOF from 127.24.71.193:52007 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 22:26:10.080533 2275 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:26:10.181430 2279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:10.181715 2279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:10.181828 2279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:10.186286 2279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:10.186371 2279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:10.186435 2279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:26:10.192454 2279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.194.keytab
--principal=oryx/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:41687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=44809
--tserver_master_addrs=127.24.71.254:36577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:10.193646 2279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:10.194747 2279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:10.205941 2285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:10.205930 2284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:10.205930 2287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:10.206142 2279 server_base.cc:1034] running on GCE node
I20260811 22:26:10.206652 2279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:10.207379 2279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:10.208595 2279 hybrid_clock.cc:648] HybridClock initialized: now 1786487170208553 us; error 53 us; skew 500 ppm
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487170, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:10.211933 2279 init.cc:377] Logged in from keytab as oryx/127.24.71.194@KRBTEST.COM (short username oryx)
I20260811 22:26:10.212950 2279 webserver.cc:460] Webserver started at http://127.24.71.194:44809/ using document root <none> and password file <none>
I20260811 22:26:10.213606 2279 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:10.213697 2279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:10.218324 2279 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:26:10.220841 2294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:10.222385 2279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:10.222527 2279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "0ea8808d91574bb6ace82dda4f44e5e0"
format_stamp: "Formatted at 2026-08-11 22:26:09 on dist-test-slave-mvvj"
server_key: "03e1127691a710f35718897734f98fc9"
server_key_iv: "8713c2dc8f7e01fa51011a75fe4f5fe7"
server_key_version: "encryptionkey@0"
I20260811 22:26:10.223117 2279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:10.245718 2279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:10.249560 2279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:10.249807 2279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:10.250514 2279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:10.251466 2279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:10.251516 2279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:10.251559 2279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:10.251616 2279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:10.262276 2279 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:41687
I20260811 22:26:10.262300 2422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:41687 every 8 connection(s)
I20260811 22:26:10.263469 2279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:26:10.264355 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 2279
I20260811 22:26:10.264905 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:38951
--local_ip_for_outbound_sockets=127.24.71.195
--tserver_master_addrs=127.24.71.254:36577
--webserver_port=40413
--webserver_interface=127.24.71.195
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.71.195
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:26:10.279102 2277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:10.265974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:37229 (local address 127.24.71.254:36577)
0811 22:26:10.266174 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.266182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.267180 (+ 998us) server_negotiation.cc:407] Connection header received
0811 22:26:10.269076 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.269079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.269136 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.269220 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.271160 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.271784 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.274035 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.274285 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.274384 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.278899 (+ 4515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:37229: BlockingRecv error: recv got EOF from 127.24.71.194:37229 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:26:10.279632 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.266383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:37229)
0811 22:26:10.266999 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:10.267062 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:10.268786 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:10.269454 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:10.269467 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:10.270090 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:10.270939 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.270967 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.272022 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.272029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.273856 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.273875 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.274494 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.274501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.274790 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.275581 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:10.275610 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:10.278633 (+ 3023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":138,"threads_started":1}
W20260811 22:26:10.280333 2423 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:36577 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.71.254:36577: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:10.287458 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.280883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:45953)
0811 22:26:10.281173 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:10.281194 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:10.281277 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:10.281555 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:10.281557 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:10.281695 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:10.281844 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.281850 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.283055 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.283058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.283958 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.283976 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.284651 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.284655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.284739 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.285322 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:10.285335 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:10.287340 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213}
W20260811 22:26:10.287662 2277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:10.281031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:45953 (local address 127.24.71.254:36577)
0811 22:26:10.281184 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.281189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.281256 (+ 67us) server_negotiation.cc:407] Connection header received
0811 22:26:10.281369 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.281373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.281427 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.281525 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.281962 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.282906 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.284090 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.284503 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.284609 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.287528 (+ 2919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:45953: BlockingRecv error: recv got EOF from 127.24.71.194:45953 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:10.294708 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.288312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:42049)
0811 22:26:10.288577 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:10.288594 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:10.288684 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:10.288928 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:10.288932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:10.289137 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:10.289250 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.289258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.290047 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.290051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.291017 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.291032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.291584 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.291588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.291695 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.292267 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:10.292284 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:10.294529 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181}
W20260811 22:26:10.294947 2277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:10.288546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:42049 (local address 127.24.71.254:36577)
0811 22:26:10.288685 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.288688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.288703 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:10.288772 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.288774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.288817 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.288888 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.289378 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.289934 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.291177 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.291448 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.291550 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.294672 (+ 3122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:42049: BlockingRecv error: recv got EOF from 127.24.71.194:42049 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:26:10.294983 2423 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:26:10.394224 2426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:10.394568 2426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:10.394671 2426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:10.399431 2426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:10.399539 2426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:10.399641 2426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:26:10.405988 2426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/krb5kdc/oryx_127.24.71.195.keytab
--principal=oryx/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:38951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=40413
--tserver_master_addrs=127.24.71.254:36577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:10.407225 2426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:10.408442 2426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:10.420111 2434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:10.420107 2432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:10.420107 2431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:10.420282 2426 server_base.cc:1034] running on GCE node
I20260811 22:26:10.420946 2426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:10.421670 2426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:10.422878 2426 hybrid_clock.cc:648] HybridClock initialized: now 1786487170422862 us; error 28 us; skew 500 ppm
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487170, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:10.425909 2426 init.cc:377] Logged in from keytab as oryx/127.24.71.195@KRBTEST.COM (short username oryx)
I20260811 22:26:10.426976 2426 webserver.cc:460] Webserver started at http://127.24.71.195:40413/ using document root <none> and password file <none>
I20260811 22:26:10.427673 2426 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:10.427805 2426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:10.432173 2426 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 22:26:10.434564 2441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:10.436172 2426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:10.436345 2426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "539e8e44e6e04514a77e687f72e5f50c"
format_stamp: "Formatted at 2026-08-11 22:26:09 on dist-test-slave-mvvj"
server_key: "5945732f693d71107ce8bdf64d742ab9"
server_key_iv: "2087f0e0c0e4049d05553ef075927076"
server_key_version: "encryptionkey@0"
I20260811 22:26:10.437009 2426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:10.457692 2426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:10.461388 2426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:10.461691 2426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:10.462380 2426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:10.463303 2426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:10.463377 2426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:10.463459 2426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:10.463511 2426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:10.473438 2426 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:38951
I20260811 22:26:10.473464 2567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:38951 every 8 connection(s)
I20260811 22:26:10.474491 2426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:10.484758 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 2426
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:26:10.487095 2277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:10.476554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:52109 (local address 127.24.71.254:36577)
0811 22:26:10.476714 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.476718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.477660 (+ 942us) server_negotiation.cc:407] Connection header received
0811 22:26:10.479114 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.479117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.479182 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.479295 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.480593 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.481123 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.482369 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.482589 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.482741 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.486942 (+ 4201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:52109: BlockingRecv error: recv got EOF from 127.24.71.195:52109 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:26:10.487928 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.476995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:52109)
0811 22:26:10.477492 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:10.477537 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:10.478883 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:10.479512 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:10.479519 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:10.479948 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:10.480436 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.480450 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.481263 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.481268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.482215 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.482226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.482803 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.482807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.482993 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.483605 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:10.483625 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:10.486786 (+ 3161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":123,"threads_started":1}
W20260811 22:26:10.488507 2568 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:36577 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.71.254:36577: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:10.495474 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.488947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:54407)
0811 22:26:10.489132 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:10.489150 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:10.489233 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:10.489522 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:10.489525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:10.489743 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:10.489867 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.489879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.490793 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.490800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.491717 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.491729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.492340 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.492344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.492426 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.493052 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:10.493065 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:10.495330 (+ 2265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 22:26:10.495659 2277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:10.489067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:54407 (local address 127.24.71.254:36577)
0811 22:26:10.489174 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.489178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.489190 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:26:10.489318 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.489321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.489364 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.489454 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.490035 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.490653 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.491892 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.492195 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.492323 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.495492 (+ 3169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:54407: BlockingRecv error: recv got EOF from 127.24.71.195:54407 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 22:26:10.497350 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.486025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.24.71.254:36577)
0811 22:26:10.486313 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.486316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.486328 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:26:10.486362 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.486364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.486401 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.486512 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.487319 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.487943 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.489164 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.489534 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.489624 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.491557 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:10.491563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:10.491581 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:10.491613 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:10.493836 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:10.494497 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:10.494506 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:10.494516 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:10.494702 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:10.495064 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:10.495074 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:10.495078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:10.495595 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:10.496400 (+ 805us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:10.496935 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 22:26:10.497190 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":103,"threads_started":1}
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:10 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:10.501374 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:10.496025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:49105)
0811 22:26:10.496143 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:10.496159 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:10.496244 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:10.496480 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:10.496483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:10.496647 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:10.496752 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.496759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.497720 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.497724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.498260 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:10.498268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.498726 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.498729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:10.498807 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.499309 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:10.499328 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:10.501280 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:26:10.501543 2277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:10.496136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:49105 (local address 127.24.71.254:36577)
0811 22:26:10.496273 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:10.496276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:10.496288 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:26:10.496328 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:10.496330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:10.496364 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:10.496429 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:10.496863 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.497598 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.498397 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:10.498611 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:10.498722 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:10.501418 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:49105: BlockingRecv error: recv got EOF from 127.24.71.195:49105 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 22:26:10.501695 2568 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 22:26:11 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:11 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:11.087536 2573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:11.081301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:49359)
0811 22:26:11.081652 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:11.081665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:11.081798 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:11.082065 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:11.082067 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:11.082251 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:11.082374 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:11.082379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.083181 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.083186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:11.084080 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:11.084093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.084690 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.084694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:11.084791 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:11.085384 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:11.085397 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:11.087351 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":166,"threads_started":1}
W20260811 22:26:11.087616 2574 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:11.081459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:49359 (local address 127.24.71.254:36577)
0811 22:26:11.081740 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:11.081743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:11.081758 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:11.081893 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:11.081895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:11.081941 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:11.082028 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:11.082494 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.083063 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.084211 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.084549 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.084653 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:11.087485 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:49359: BlockingRecv error: recv got EOF from 127.24.71.193:49359 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":114,"threads_started":1}
Aug 11 22:26:11 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:11 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:11.302304 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:11.295942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:41181)
0811 22:26:11.296234 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:11.296249 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:11.296361 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:11.296639 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:11.296641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:11.296840 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:11.296940 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:11.296945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.297750 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.297753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:11.298349 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:11.298359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.298913 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.298916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:11.298985 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:11.299627 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:11.299640 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:11.302115 (+ 2475us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":110,"threads_started":1}
W20260811 22:26:11.302510 2574 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:11.295959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:41181 (local address 127.24.71.254:36577)
0811 22:26:11.296103 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:11.296107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:11.296429 (+ 322us) server_negotiation.cc:407] Connection header received
0811 22:26:11.296482 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:11.296484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:11.296532 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:11.296596 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:11.297048 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.297615 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.298541 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.298784 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.298883 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:11.302294 (+ 3411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:41181: BlockingRecv error: recv got EOF from 127.24.71.194:41181 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 22:26:11 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:11 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:11.508929 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:11.502430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:60761)
0811 22:26:11.502709 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:11.502724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:11.502851 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:11.503143 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:11.503146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:11.503380 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:11.503493 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:11.503501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.504472 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.504476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:11.505154 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:11.505166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.505639 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.505642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:11.505713 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:11.506371 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:11.506396 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:11.508795 (+ 2399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":113,"threads_started":1}
W20260811 22:26:11.509099 2574 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:11.502511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:60761 (local address 127.24.71.254:36577)
0811 22:26:11.502658 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:11.502662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:11.502840 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:26:11.502928 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:11.502930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:11.502981 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:11.503064 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:11.503633 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.504268 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.505288 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:11.505519 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:11.505604 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:11.508935 (+ 3331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:60761: BlockingRecv error: recv got EOF from 127.24.71.195:60761 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 22:26:12 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:12 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:12.095245 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:12.088488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:41197)
0811 22:26:12.088908 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:12.088927 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:12.089044 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:12.089438 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:12.089441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:12.089643 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:12.089755 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:12.089761 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.090713 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.090716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:12.091366 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:12.091378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.092006 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.092009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:12.092083 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:12.092717 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:12.092730 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:12.095097 (+ 2367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":104,"threads_started":1}
W20260811 22:26:12.095603 2578 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:12.088633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:41197 (local address 127.24.71.254:36577)
0811 22:26:12.088905 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:12.088909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:12.089014 (+ 105us) server_negotiation.cc:407] Connection header received
0811 22:26:12.089168 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:12.089170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:12.089220 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:12.089307 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:12.089915 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.090574 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.091529 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.091780 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.092000 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:12.095438 (+ 3438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:41197: BlockingRecv error: recv got EOF from 127.24.71.193:41197 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":79,"threads_started":1}
Aug 11 22:26:12 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:12 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:12.309392 2579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:12.303220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:45695)
0811 22:26:12.303604 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:12.303623 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:12.303777 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:12.304108 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:12.304111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:12.304370 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:12.304474 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:12.304482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.305263 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.305268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:12.305823 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:12.305831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.306319 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.306323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:12.306436 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:12.307055 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:12.307067 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:12.309214 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":127,"threads_started":1}
W20260811 22:26:12.309520 2578 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:12.303384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:45695 (local address 127.24.71.254:36577)
0811 22:26:12.303608 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:12.303616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:12.303713 (+ 97us) server_negotiation.cc:407] Connection header received
0811 22:26:12.303922 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:12.303925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:12.303988 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:12.304080 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:12.304605 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.305146 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.305949 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.306187 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.306300 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:12.309359 (+ 3059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:45695: BlockingRecv error: recv got EOF from 127.24.71.194:45695 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Aug 11 22:26:12 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:12 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:12.516613 2580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:12.509877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:56319)
0811 22:26:12.510340 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:12.510360 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:12.510495 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:12.510813 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:12.510817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:12.511133 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:12.511310 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:12.511319 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.512292 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.512296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:12.512946 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:12.512956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.513452 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.513455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:12.513568 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:12.514112 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:12.514127 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:12.516469 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":126,"threads_started":1}
W20260811 22:26:12.516767 2578 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:12.510087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:56319 (local address 127.24.71.254:36577)
0811 22:26:12.510261 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:12.510267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:12.510480 (+ 213us) server_negotiation.cc:407] Connection header received
0811 22:26:12.510596 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:12.510599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:12.510664 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:12.510742 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:12.511509 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.512156 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.513074 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:12.513294 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:12.513380 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:12.516635 (+ 3255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:56319: BlockingRecv error: recv got EOF from 127.24.71.195:56319 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 22:26:13 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:13 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:13.103134 2581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:13.096207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:58049)
0811 22:26:13.096534 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:13.096555 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:13.096665 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:13.097017 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:13.097020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:13.097223 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:13.097345 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:13.097352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.098202 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.098204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:13.098863 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:13.098873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.099344 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.099347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:13.099416 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:13.100116 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:13.100134 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:13.102914 (+ 2780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":114,"threads_started":1}
W20260811 22:26:13.103188 2582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:13.096284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:58049 (local address 127.24.71.254:36577)
0811 22:26:13.096618 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:13.096622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:13.096639 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:26:13.096779 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:13.096782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:13.096833 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:13.096935 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:13.097529 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.098082 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.098993 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.099226 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.099309 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:13.103062 (+ 3753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:58049: BlockingRecv error: recv got EOF from 127.24.71.193:58049 (error 108)
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":150,"threads_started":1}
Aug 11 22:26:13 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:13 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:13.317217 2583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:13.310529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:47641)
0811 22:26:13.310960 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:13.310974 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:13.311132 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:13.311486 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:13.311490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:13.311755 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:13.311938 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:13.311947 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.312806 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.312811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:13.313440 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:13.313449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.314020 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.314025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:13.314122 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:13.314759 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:13.314773 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:13.317061 (+ 2288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":127,"threads_started":1}
W20260811 22:26:13.317447 2582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:13.310577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:47641 (local address 127.24.71.254:36577)
0811 22:26:13.310754 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:13.310760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:13.311057 (+ 297us) server_negotiation.cc:407] Connection header received
0811 22:26:13.311287 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:13.311290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:13.311344 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:13.311459 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:13.312076 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.312680 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.313603 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.313874 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.313998 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:13.317247 (+ 3249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:47641: BlockingRecv error: recv got EOF from 127.24.71.194:47641 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 22:26:13 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:13 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:13.523679 2584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:13.517616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:42203)
0811 22:26:13.518083 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:13.518097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:13.518237 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:13.518545 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:13.518548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:13.518760 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:13.518863 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:13.518869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.519665 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.519670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:13.520252 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:13.520261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.520732 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.520736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:13.520824 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:13.521417 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:13.521430 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:13.523420 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":133,"threads_started":1}
W20260811 22:26:13.523685 2582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:13.517691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:42203 (local address 127.24.71.254:36577)
0811 22:26:13.517870 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:13.517874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:13.518177 (+ 303us) server_negotiation.cc:407] Connection header received
0811 22:26:13.518365 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:13.518368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:13.518424 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:13.518514 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:13.519027 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.519544 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.520378 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:13.520612 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:13.520726 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:13.523569 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:42203: BlockingRecv error: recv got EOF from 127.24.71.195:42203 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 22:26:14 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:14 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:14.120752 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:14.104193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:37073)
0811 22:26:14.104608 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:14.104624 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:14.104728 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:14.105026 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:14.105028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:14.105215 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:14.105330 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:14.105335 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.106147 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.106149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:14.106788 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:14.106801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.107416 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.107420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:14.107510 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:14.117720 (+ 10210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:14.117738 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:14.120559 (+ 2821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":97,"threads_started":1}
W20260811 22:26:14.121001 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:14.104364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:37073 (local address 127.24.71.254:36577)
0811 22:26:14.104671 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:14.104675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:14.104691 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:26:14.104830 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:14.104833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:14.104878 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:14.104965 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:14.105482 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.106038 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.106965 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.107278 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.107406 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:14.120726 (+ 13320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:37073: BlockingRecv error: recv got EOF from 127.24.71.193:37073 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":75,"threads_started":1}
Aug 11 22:26:14 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:14 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:14.324818 2587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:14.318365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:49941)
0811 22:26:14.318733 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:14.318760 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:14.318880 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:14.319338 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:14.319342 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:14.319572 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:14.319689 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:14.319696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.320655 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.320661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:14.321295 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:14.321306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.321841 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.321845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:14.321962 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:14.322498 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:14.322511 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:14.324626 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
W20260811 22:26:14.324899 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:14.318558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:49941 (local address 127.24.71.254:36577)
0811 22:26:14.318722 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:14.318726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:14.318842 (+ 116us) server_negotiation.cc:407] Connection header received
0811 22:26:14.319076 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:14.319080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:14.319140 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:14.319233 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:14.319885 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.320516 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.321440 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.321699 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.321827 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:14.324778 (+ 2951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:49941: BlockingRecv error: recv got EOF from 127.24.71.194:49941 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 22:26:14 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:14 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:14.530877 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:14.524690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:40279)
0811 22:26:14.524987 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:14.525003 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:14.525100 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:14.525404 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:14.525406 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:14.525596 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:14.525696 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:14.525702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.526623 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.526626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:14.527272 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:14.527284 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.527901 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.527908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:14.527986 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:14.528522 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:14.528534 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:14.530696 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":89,"threads_started":1}
W20260811 22:26:14.531204 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:14.524837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:40279 (local address 127.24.71.254:36577)
0811 22:26:14.525033 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:14.525037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:14.525052 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:14.525194 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:14.525197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:14.525246 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:14.525316 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:14.525840 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.526472 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.527413 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:14.527716 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:14.527805 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:14.530878 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:40279: BlockingRecv error: recv got EOF from 127.24.71.195:40279 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 22:26:15 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:15 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:15.129161 2589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:15.122179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:36453)
0811 22:26:15.122500 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:15.122519 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:15.122653 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:15.122975 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:15.122978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:15.123183 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:15.123309 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:15.123315 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.124328 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.124331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:15.125023 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:15.125037 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.125736 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.125741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:15.125841 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:15.126715 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:15.126749 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:15.129028 (+ 2279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"spinlock_wait_cycles":7552,"thread_start_us":128,"threads_started":1}
W20260811 22:26:15.129344 2590 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:15.122173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:36453 (local address 127.24.71.254:36577)
0811 22:26:15.122490 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:15.122494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:15.122663 (+ 169us) server_negotiation.cc:407] Connection header received
0811 22:26:15.122727 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:15.122730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:15.122781 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:15.122883 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:15.123508 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.124162 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.125162 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.125583 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.125701 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:15.129197 (+ 3496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:36453: BlockingRecv error: recv got EOF from 127.24.71.193:36453 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":132,"threads_started":1}
Aug 11 22:26:15 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:15 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:15.331545 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:15.325741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:37477)
0811 22:26:15.326077 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:15.326091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:15.326185 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:15.326441 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:15.326444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:15.326652 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:15.326753 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:15.326758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.327530 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.327533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:15.328193 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:15.328203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.328727 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.328731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:15.328800 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:15.329425 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:15.329437 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:15.331359 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":113,"threads_started":1}
W20260811 22:26:15.331830 2590 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:15.325900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:37477 (local address 127.24.71.254:36577)
0811 22:26:15.326079 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:15.326083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:15.326171 (+ 88us) server_negotiation.cc:407] Connection header received
0811 22:26:15.326262 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:15.326264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:15.326310 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:15.326385 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:15.326865 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.327423 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.328344 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.328573 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.328670 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:15.331708 (+ 3038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:37477: BlockingRecv error: recv got EOF from 127.24.71.194:37477 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 22:26:15 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:15 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:15.538345 2592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:15.531932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:36197)
0811 22:26:15.532380 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:15.532396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:15.532550 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:15.532900 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:15.532903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:15.533143 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:15.533241 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:15.533248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.534063 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.534069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:15.534860 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:15.534872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.535496 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.535500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:15.535597 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:15.536243 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:15.536255 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:15.538213 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":114,"threads_started":1}
W20260811 22:26:15.538578 2590 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:15.532072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:36197 (local address 127.24.71.254:36577)
0811 22:26:15.532297 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:15.532302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:15.532485 (+ 183us) server_negotiation.cc:407] Connection header received
0811 22:26:15.532673 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:15.532681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:15.532732 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:15.532879 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:15.533379 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.533930 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.534991 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:15.535307 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:15.535477 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:15.538421 (+ 2944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:36197: BlockingRecv error: recv got EOF from 127.24.71.195:36197 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 22:26:16 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:16 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:16.136356 2593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:16.130111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:35119)
0811 22:26:16.130414 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:16.130428 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:16.130522 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:16.130793 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:16.130796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:16.130992 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:16.131098 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:16.131104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.132059 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.132062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:16.132680 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:16.132691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.133241 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.133244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:16.133313 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:16.133936 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:16.133955 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:16.136155 (+ 2200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
W20260811 22:26:16.136476 2594 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:16.130235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:35119 (local address 127.24.71.254:36577)
0811 22:26:16.130513 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:16.130517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:16.130531 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:16.130609 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:16.130612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:16.130667 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:16.130740 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:16.131249 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.131907 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.132838 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.133085 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.133182 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:16.136333 (+ 3151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:35119: BlockingRecv error: recv got EOF from 127.24.71.193:35119 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
Aug 11 22:26:16 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:16 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:16.338685 2595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:16.332417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:45255)
0811 22:26:16.332722 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:16.332736 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:16.332835 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:16.333232 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:16.333235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:16.333463 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:16.333568 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:16.333575 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.334444 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.334447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:16.335063 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:16.335076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.335639 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.335641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:16.335711 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:16.336385 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:16.336400 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:16.338556 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":106,"threads_started":1}
W20260811 22:26:16.338918 2594 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:16.332610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:45255 (local address 127.24.71.254:36577)
0811 22:26:16.332780 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:16.332784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:16.332800 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:26:16.332988 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:16.332991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:16.333065 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:16.333150 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:16.333725 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.334288 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.335230 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.335503 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.335612 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:16.338767 (+ 3155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:45255: BlockingRecv error: recv got EOF from 127.24.71.194:45255 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 22:26:16 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:16 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:16.546236 2596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:16.539481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:45309)
0811 22:26:16.540018 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:16.540037 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:16.540180 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:16.540563 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:16.540567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:16.540831 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:16.540989 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:16.541002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.541895 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.541900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:16.542615 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:16.542627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.543109 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.543112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:16.543174 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:16.543728 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:16.543742 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:16.546102 (+ 2360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":150,"threads_started":1}
W20260811 22:26:16.546540 2594 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:16.539616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:45309 (local address 127.24.71.254:36577)
0811 22:26:16.539820 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:16.539854 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:16.540167 (+ 313us) server_negotiation.cc:407] Connection header received
0811 22:26:16.540291 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:16.540297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:16.540370 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:16.540489 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:16.541152 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.541734 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.542747 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:16.542984 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:16.543075 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:16.546404 (+ 3329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:45309: BlockingRecv error: recv got EOF from 127.24.71.195:45309 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 22:26:17 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:17 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:17.143427 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:17.137408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:55265)
0811 22:26:17.137784 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:17.137798 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:17.137901 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:17.138195 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:17.138197 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:17.138402 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:17.138501 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:17.138506 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.139426 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.139429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:17.140062 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:17.140073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.140608 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.140611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:17.140672 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:17.141209 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:17.141221 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:17.143303 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":136,"threads_started":1}
W20260811 22:26:17.143664 2598 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:17.137487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:55265 (local address 127.24.71.254:36577)
0811 22:26:17.137756 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:17.137760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:17.137892 (+ 132us) server_negotiation.cc:407] Connection header received
0811 22:26:17.137974 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:17.137976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:17.138036 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:17.138124 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:17.138654 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.139287 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.140204 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.140487 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.140595 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:17.143499 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:55265: BlockingRecv error: recv got EOF from 127.24.71.193:55265 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":80,"threads_started":1}
Aug 11 22:26:17 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:17 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:17.346692 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:17.339896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:42303)
0811 22:26:17.340292 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:17.340306 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:17.340435 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:17.340701 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:17.340703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:17.340930 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:17.341032 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:17.341038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.342024 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.342027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:17.342727 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:17.342743 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.343482 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.343486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:17.343561 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:17.344175 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:17.344188 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:17.346510 (+ 2322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":138,"threads_started":1}
W20260811 22:26:17.346755 2598 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:17.339939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:42303 (local address 127.24.71.254:36577)
0811 22:26:17.340132 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:17.340137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:17.340410 (+ 273us) server_negotiation.cc:407] Connection header received
0811 22:26:17.340528 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:17.340530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:17.340581 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:17.340656 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:17.341253 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.341863 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.342882 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.343159 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.343269 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:17.346645 (+ 3376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:42303: BlockingRecv error: recv got EOF from 127.24.71.194:42303 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 22:26:17 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:17 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:17.554390 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:17.547535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:38059)
0811 22:26:17.547983 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:17.548000 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:17.548122 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:17.548398 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:17.548401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:17.548665 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:17.548767 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:17.548784 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.549708 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.549711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:17.550323 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:17.550333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.550967 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.550970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:17.551043 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:17.551807 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:17.551824 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:17.554194 (+ 2370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":131,"threads_started":1}
W20260811 22:26:17.554559 2598 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:17.547603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:38059 (local address 127.24.71.254:36577)
0811 22:26:17.547781 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:17.547787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:17.548089 (+ 302us) server_negotiation.cc:407] Connection header received
0811 22:26:17.548216 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:17.548219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:17.548275 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:17.548349 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:17.548947 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.549571 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.550585 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:17.550816 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:17.550911 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:17.554351 (+ 3440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:38059: BlockingRecv error: recv got EOF from 127.24.71.195:38059 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 22:26:18 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:18 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:18.151388 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:18.144412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:33533)
0811 22:26:18.144753 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:18.144767 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:18.144863 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:18.145228 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:18.145230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:18.145421 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:18.145521 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:18.145526 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.146405 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.146408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:18.147056 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:18.147066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.147609 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.147617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:18.147710 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:18.148490 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:18.148575 (+ 85us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:18.151259 (+ 2684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":137,"threads_started":1}
W20260811 22:26:18.151724 2602 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:18.144588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:33533 (local address 127.24.71.254:36577)
0811 22:26:18.144858 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:18.144862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:18.144877 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:18.145036 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:18.145040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:18.145096 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:18.145190 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:18.145638 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.146256 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.147186 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.147432 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.147521 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:18.151572 (+ 4051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:33533: BlockingRecv error: recv got EOF from 127.24.71.193:33533 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":63,"threads_started":1}
Aug 11 22:26:18 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:18 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:18.354432 2603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:18.347672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:58727)
0811 22:26:18.347998 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:18.348012 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:18.348125 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:18.348460 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:18.348463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:18.348707 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:18.348816 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:18.348823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.349969 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.349972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:18.350599 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:18.350611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.351154 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.351158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:18.351251 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:18.351952 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:18.351968 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:18.354304 (+ 2336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":104,"threads_started":1}
W20260811 22:26:18.354673 2602 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:18.347805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:58727 (local address 127.24.71.254:36577)
0811 22:26:18.348029 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:18.348033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:18.348071 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:26:18.348226 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:18.348228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:18.348279 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:18.348367 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:18.348974 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.349829 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.350734 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.350988 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.351078 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:18.354553 (+ 3475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:58727: BlockingRecv error: recv got EOF from 127.24.71.194:58727 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 11 22:26:18 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:18 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:18.562309 2604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:18.555559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:46469)
0811 22:26:18.555952 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:18.555970 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:18.556112 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:18.556429 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:18.556432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:18.556664 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:18.556772 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:18.556778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.557646 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.557650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:18.558243 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:18.558253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.558836 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.558839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:18.558909 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:18.559533 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:18.559545 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:18.562174 (+ 2629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
W20260811 22:26:18.562620 2602 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:18.555729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:46469 (local address 127.24.71.254:36577)
0811 22:26:18.555957 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:18.555961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:18.556112 (+ 151us) server_negotiation.cc:407] Connection header received
0811 22:26:18.556191 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:18.556194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:18.556249 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:18.556400 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:18.556919 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.557512 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.558398 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:18.558696 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:18.558797 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:18.562484 (+ 3687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:46469: BlockingRecv error: recv got EOF from 127.24.71.195:46469 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 22:26:19 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:19 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:19.163498 2605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:19.152408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:59959)
0811 22:26:19.152737 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:19.152755 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:19.152919 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:19.153236 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:19.153239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:19.153462 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:19.153561 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:19.153567 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.154348 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.154353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:19.154967 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:19.154976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.155459 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.155463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:19.155552 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:19.161072 (+ 5520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:19.161088 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:19.163324 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":89,"threads_started":1}
W20260811 22:26:19.163674 2606 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:19.152501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:59959 (local address 127.24.71.254:36577)
0811 22:26:19.152819 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:19.152824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:19.152838 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:19.153038 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:19.153042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:19.153098 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:19.153207 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:19.153686 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.154233 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.155095 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.155330 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.155452 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:19.163514 (+ 8062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:59959: BlockingRecv error: recv got EOF from 127.24.71.193:59959 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":75,"threads_started":1}
Aug 11 22:26:19 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:19 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:19.361810 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:19.355374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:52139)
0811 22:26:19.355711 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:19.355726 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:19.355896 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:19.356214 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:19.356217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:19.356461 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:19.356565 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:19.356571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.357535 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.357539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:19.358320 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:19.358332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.358823 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.358827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:19.358918 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:19.359457 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:19.359470 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:19.361686 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":81,"threads_started":1}
W20260811 22:26:19.362032 2606 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:19.355562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:52139 (local address 127.24.71.254:36577)
0811 22:26:19.355739 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:19.355743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:19.355891 (+ 148us) server_negotiation.cc:407] Connection header received
0811 22:26:19.355983 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:19.355986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:19.356044 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:19.356134 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:19.356723 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.357305 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.358448 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.358696 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.358814 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:19.361880 (+ 3066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:52139: BlockingRecv error: recv got EOF from 127.24.71.194:52139 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 22:26:19 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:19 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:19.569962 2608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:19.563480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:54349)
0811 22:26:19.563925 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:19.563942 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:19.564136 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:19.564458 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:19.564461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:19.564699 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:19.564796 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:19.564802 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.565661 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.565666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:19.566360 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:19.566370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.566833 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.566837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:19.566923 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:19.567457 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:19.567471 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:19.569767 (+ 2296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":114,"threads_started":1}
W20260811 22:26:19.570050 2606 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:19.563599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:54349 (local address 127.24.71.254:36577)
0811 22:26:19.563889 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:19.563894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:19.564068 (+ 174us) server_negotiation.cc:407] Connection header received
0811 22:26:19.564248 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:19.564251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:19.564307 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:19.564412 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:19.564925 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.565547 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.566490 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:19.566717 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:19.566826 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:19.569922 (+ 3096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:54349: BlockingRecv error: recv got EOF from 127.24.71.195:54349 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
Aug 11 22:26:20 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:20 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:20.170639 2609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:20.164688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:38809)
0811 22:26:20.165157 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:20.165172 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:20.165293 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:20.165690 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:20.165696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:20.165915 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:20.166019 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:20.166025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.166848 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.166851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:20.167493 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:20.167504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.168001 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.168004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:20.168079 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:20.168590 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:20.168602 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:20.170522 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":150,"threads_started":1}
W20260811 22:26:20.170881 2610 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:20.164711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:38809 (local address 127.24.71.254:36577)
0811 22:26:20.165068 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:20.165072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:20.165249 (+ 177us) server_negotiation.cc:407] Connection header received
0811 22:26:20.165403 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:20.165406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:20.165452 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:20.165624 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:20.166140 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.166706 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.167625 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.167876 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.167958 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:20.170741 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:38809: BlockingRecv error: recv got EOF from 127.24.71.193:38809 (error 108)
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":114,"threads_started":1}
Aug 11 22:26:20 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:20 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:20.369119 2611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:20.362843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:59883)
0811 22:26:20.363178 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:20.363193 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:20.363305 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:20.363606 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:20.363609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:20.363896 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:20.364006 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:20.364013 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.364971 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.364974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:20.365578 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:20.365588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.366086 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.366088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:20.366154 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:20.366701 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:20.366713 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:20.368975 (+ 2262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":125,"threads_started":1}
W20260811 22:26:20.369344 2610 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:20.362940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:59883 (local address 127.24.71.254:36577)
0811 22:26:20.363108 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:20.363114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:20.363307 (+ 193us) server_negotiation.cc:407] Connection header received
0811 22:26:20.363395 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:20.363397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:20.363447 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:20.363525 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:20.364154 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.364792 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.365718 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.365962 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.366062 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:20.369200 (+ 3138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:59883: BlockingRecv error: recv got EOF from 127.24.71.194:59883 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 22:26:20 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:20 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:20.576918 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:20.571025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:33209)
0811 22:26:20.571361 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:20.571376 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:20.571467 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:20.571773 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:20.571775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:20.572052 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:20.572153 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:20.572159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.572969 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.572971 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:20.573533 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:20.573541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.574027 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.574030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:20.574091 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:20.574598 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:20.574610 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:20.576756 (+ 2146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":104,"threads_started":1}
W20260811 22:26:20.577203 2610 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:20.571237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:33209 (local address 127.24.71.254:36577)
0811 22:26:20.571399 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:20.571402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:20.571420 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:26:20.571591 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:20.571594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:20.571648 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:20.571729 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:20.572287 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.572863 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.573647 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:20.573903 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:20.573990 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:20.577020 (+ 3030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:33209: BlockingRecv error: recv got EOF from 127.24.71.195:33209 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 22:26:21 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:21 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:21.178009 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:21.171584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:45889)
0811 22:26:21.171911 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:21.171935 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:21.172038 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:21.172331 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:21.172333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:21.172517 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:21.172627 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:21.172632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.173789 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.173794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:21.174501 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:21.174514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.175122 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.175126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:21.175193 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:21.175787 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:21.175799 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:21.177861 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":99,"threads_started":1}
W20260811 22:26:21.178241 2614 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:21.171673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:45889 (local address 127.24.71.254:36577)
0811 22:26:21.171981 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:21.171985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:21.172002 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:26:21.172133 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:21.172136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:21.172195 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:21.172288 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:21.172776 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.173623 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.174631 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.175003 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.175102 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:21.178047 (+ 2945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:45889: BlockingRecv error: recv got EOF from 127.24.71.193:45889 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":109,"threads_started":1}
Aug 11 22:26:21 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:21 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:21.375995 2617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:21.370073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:59825)
0811 22:26:21.370361 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:21.370377 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:21.370520 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:21.370831 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:21.370833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:21.371031 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:21.371130 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:21.371136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.371923 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.371927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:21.372488 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:21.372497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.372951 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.372954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:21.373016 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:21.373558 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:21.373570 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:21.375795 (+ 2225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":97,"threads_started":1}
W20260811 22:26:21.376171 2614 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:21.370090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:59825 (local address 127.24.71.254:36577)
0811 22:26:21.370263 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:21.370268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:21.370500 (+ 232us) server_negotiation.cc:407] Connection header received
0811 22:26:21.370663 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:21.370665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:21.370716 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:21.370784 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:21.371240 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.371766 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.372618 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.372841 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.372926 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:21.375996 (+ 3070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:59825: BlockingRecv error: recv got EOF from 127.24.71.194:59825 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 22:26:21 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:21 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:21.583949 2618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:21.577880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:48279)
0811 22:26:21.578186 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:21.578200 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:21.578329 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:21.578615 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:21.578618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:21.578815 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:21.578918 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:21.578961 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.579722 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.579725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:21.580416 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:21.580429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.580866 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.580869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:21.580965 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:21.581475 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:21.581488 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:21.583729 (+ 2241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":98,"threads_started":1}
W20260811 22:26:21.584020 2614 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:21.578026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:48279 (local address 127.24.71.254:36577)
0811 22:26:21.578195 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:21.578199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:21.578272 (+ 73us) server_negotiation.cc:407] Connection header received
0811 22:26:21.578415 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:21.578417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:21.578498 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:21.578570 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:21.579064 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.579613 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.580543 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:21.580758 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:21.580835 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:21.583884 (+ 3049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:48279: BlockingRecv error: recv got EOF from 127.24.71.195:48279 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 22:26:22 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:22 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:22.184932 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:22.178887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:42347)
0811 22:26:22.179213 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:22.179227 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:22.179336 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:22.179667 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:22.179670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:22.179904 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:22.180032 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:22.180039 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.180857 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.180860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:22.181501 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:22.181511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.181954 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.181957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:22.182032 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:22.182598 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:22.182611 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:22.184812 (+ 2201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":121,"threads_started":1}
W20260811 22:26:22.185166 2620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:22.179049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:42347 (local address 127.24.71.254:36577)
0811 22:26:22.179280 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:22.179284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:22.179299 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:22.179432 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:22.179434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:22.179481 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:22.179561 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:22.180194 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.180752 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.181612 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.181830 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.181906 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:22.184999 (+ 3093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:42347: BlockingRecv error: recv got EOF from 127.24.71.193:42347 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":63,"threads_started":1}
Aug 11 22:26:22 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:22 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:22.383558 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:22.377113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:45745)
0811 22:26:22.377452 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:22.377468 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:22.377585 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:22.377876 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:22.377878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:22.378110 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:22.378216 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:22.378222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.379052 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.379055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:22.379695 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:22.379706 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.380316 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.380319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:22.380391 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:22.381024 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:22.381040 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:22.383373 (+ 2333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":125,"threads_started":1}
W20260811 22:26:22.383746 2620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:22.377243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:45745 (local address 127.24.71.254:36577)
0811 22:26:22.377438 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:22.377442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:22.377558 (+ 116us) server_negotiation.cc:407] Connection header received
0811 22:26:22.377681 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:22.377684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:22.377735 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:22.377825 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:22.378354 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.378934 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.379816 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.380148 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.380239 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:22.383604 (+ 3365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:45745: BlockingRecv error: recv got EOF from 127.24.71.194:45745 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 22:26:22 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:22 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:22.591094 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:22.584875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:39489)
0811 22:26:22.585158 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:22.585171 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:22.585275 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:22.585542 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:22.585544 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:22.585734 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:22.585832 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:22.585837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.586807 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.586812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:22.587713 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:22.587732 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.588268 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.588271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:22.588395 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:22.588897 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:22.588910 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:22.590963 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":105,"threads_started":1}
W20260811 22:26:22.591235 2620 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:22.584962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:39489 (local address 127.24.71.254:36577)
0811 22:26:22.585107 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:22.585111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:22.585242 (+ 131us) server_negotiation.cc:407] Connection header received
0811 22:26:22.585358 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:22.585361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:22.585411 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:22.585489 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:22.585959 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.586610 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.587855 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:22.588161 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:22.588240 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:22.591109 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:39489: BlockingRecv error: recv got EOF from 127.24.71.195:39489 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 22:26:23 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:23 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:23.191978 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:23.185871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:37523)
0811 22:26:23.186150 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:23.186166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:23.186279 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:23.186575 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:23.186577 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:23.186771 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:23.186879 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:23.186885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.187710 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.187713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:23.188412 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:23.188425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.188963 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.188966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:23.189035 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:23.189697 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:23.189715 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:23.191781 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":97,"threads_started":1}
W20260811 22:26:23.192163 2624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:23.186007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:37523 (local address 127.24.71.254:36577)
0811 22:26:23.186252 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:23.186256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:23.186270 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:23.186379 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:23.186382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:23.186441 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:23.186529 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:23.186999 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.187578 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.188558 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.188806 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.188912 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:23.191990 (+ 3078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:37523: BlockingRecv error: recv got EOF from 127.24.71.193:37523 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
Aug 11 22:26:23 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:23 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:23.390895 2625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:23.384486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:56485)
0811 22:26:23.384807 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:23.384821 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:23.384921 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:23.385249 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:23.385251 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:23.385439 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:23.385569 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:23.385575 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.386449 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.386453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:23.387106 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:23.387117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.387622 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.387625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:23.387696 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:23.388304 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:23.388318 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:23.390689 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":111,"threads_started":1}
W20260811 22:26:23.391100 2624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:23.384716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:56485 (local address 127.24.71.254:36577)
0811 22:26:23.384870 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:23.384874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:23.384890 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:26:23.385061 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:23.385064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:23.385113 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:23.385198 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:23.385731 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.386276 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.387256 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.387500 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.387587 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:23.390920 (+ 3333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:56485: BlockingRecv error: recv got EOF from 127.24.71.194:56485 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 22:26:23 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:23 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:23.597982 2626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:23.591993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:42647)
0811 22:26:23.592348 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:23.592371 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:23.592512 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:23.592840 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:23.592843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:23.593075 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:23.593172 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:23.593179 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.593973 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.593978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:23.594546 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:23.594555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.595049 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.595052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:23.595139 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:23.595702 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:23.595715 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:23.597791 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":105,"threads_started":1}
W20260811 22:26:23.598061 2624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:23.592134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:42647 (local address 127.24.71.254:36577)
0811 22:26:23.592313 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:23.592318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:23.592449 (+ 131us) server_negotiation.cc:407] Connection header received
0811 22:26:23.592657 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:23.592661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:23.592721 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:23.592817 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:23.593309 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.593853 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.594693 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:23.594924 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:23.595042 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:23.597930 (+ 2888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:42647: BlockingRecv error: recv got EOF from 127.24.71.195:42647 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 22:26:24 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:24 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:24.203478 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:24.192912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:47833)
0811 22:26:24.193228 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:24.193245 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:24.193348 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:24.193617 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:24.193619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:24.193824 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:24.193931 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:24.193937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.194807 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.194810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:24.195471 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:24.195481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.196036 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.196039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:24.196112 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:24.201103 (+ 4991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:24.201119 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:24.203294 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":101,"threads_started":1}
W20260811 22:26:24.203811 2628 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:24.192922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:47833 (local address 127.24.71.254:36577)
0811 22:26:24.193277 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:24.193281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:24.193295 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:26:24.193443 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:24.193445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:24.193491 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:24.193567 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:24.194081 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.194672 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.195618 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.195898 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.195990 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:24.203581 (+ 7591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:47833: BlockingRecv error: recv got EOF from 127.24.71.193:47833 (error 108)
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":60,"threads_started":1}
Aug 11 22:26:24 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:24 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:24.398345 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:24.391938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:36737)
0811 22:26:24.392229 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:24.392244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:24.392360 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:24.392605 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:24.392607 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:24.392791 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:24.392913 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:24.392918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.393749 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.393753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:24.394445 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:24.394457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.394964 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.394967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:24.395043 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:24.395662 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:24.395675 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:24.398135 (+ 2460us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":109,"threads_started":1}
W20260811 22:26:24.398459 2628 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:24.392034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:36737 (local address 127.24.71.254:36577)
0811 22:26:24.392195 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:24.392199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:24.392301 (+ 102us) server_negotiation.cc:407] Connection header received
0811 22:26:24.392440 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:24.392442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:24.392488 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:24.392563 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:24.393050 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.393593 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.394602 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.394837 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.394931 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:24.398320 (+ 3389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:36737: BlockingRecv error: recv got EOF from 127.24.71.194:36737 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 22:26:24 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:24 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.195@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:24.605036 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:24.598935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.195:36539)
0811 22:26:24.599267 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:24.599283 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:24.599388 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:24.599651 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:24.599653 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:24.599877 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:24.600020 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:24.600030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.600892 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.600895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:24.601523 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:24.601534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.602033 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.602036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:24.602105 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:24.602684 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:24.602696 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:24.604836 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":142,"threads_started":1}
W20260811 22:26:24.605131 2628 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:24.599002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:36539 (local address 127.24.71.254:36577)
0811 22:26:24.599169 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:24.599172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:24.599365 (+ 193us) server_negotiation.cc:407] Connection header received
0811 22:26:24.599475 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:24.599477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:24.599533 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:24.599614 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:24.600164 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.600750 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.601665 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:24.601895 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:24.601984 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:24.604992 (+ 3008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.195:36539: BlockingRecv error: recv got EOF from 127.24.71.195:36539 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 22:26:25 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:25 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.193@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:25.210678 2631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:25.204744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.193:47177)
0811 22:26:25.205054 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:25.205072 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:25.205184 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:25.205506 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:25.205508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:25.205701 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:25.205801 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:25.205806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.206582 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.206584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:25.207128 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:25.207135 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.207647 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.207650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:25.207714 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:25.208318 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:25.208337 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:25.210546 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
W20260811 22:26:25.210911 2632 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:25.204916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:47177 (local address 127.24.71.254:36577)
0811 22:26:25.205209 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:25.205213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:25.205246 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:26:25.205293 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:25.205295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:25.205337 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:25.205467 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:25.205909 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.206477 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.207259 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.207530 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.207627 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:25.210770 (+ 3143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.193:47177: BlockingRecv error: recv got EOF from 127.24.71.193:47177 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":86,"threads_started":1}
Aug 11 22:26:25 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:26:25 dist-test-slave-mvvj krb5kdc[1485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.71.194@KRBTEST.COM for oryx/127.24.71.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:26:25.406152 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:25.399696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:36577 (local address 127.24.71.194:34421)
0811 22:26:25.400152 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:25.400172 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:25.400316 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:25.400618 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:25.400621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:25.400833 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:25.400938 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:25.400945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.401759 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.401764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:25.402346 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:25.402354 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.402920 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.402925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:25.403023 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:25.403654 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:25.403669 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:25.406035 (+ 2366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.71.254:36577: Server oryx/127.24.71.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":145,"threads_started":1}
W20260811 22:26:25.406355 2632 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:25.399895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:34421 (local address 127.24.71.254:36577)
0811 22:26:25.400109 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:25.400114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:25.400256 (+ 142us) server_negotiation.cc:407] Connection header received
0811 22:26:25.400445 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:25.400448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:25.400503 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:25.400596 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:25.401068 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.401638 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.402481 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.402772 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.402908 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:25.406209 (+ 3301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.71.194:34421: BlockingRecv error: recv got EOF from 127.24.71.194:34421 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:26:25.489355 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 2117
I20260811 22:26:25.498528 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 2279
I20260811 22:26:25.505599 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 2426
I20260811 22:26:25.513808 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 2044
2026-08-11T22:26:25Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18275 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T22:26:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:25Z Disabled control of system clock
I20260811 22:26:25.550304 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:46093
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:46093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:26:25.675709 2637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:25.676057 2637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:25.676096 2637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:25.680243 2637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:25.680323 2637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:25.680341 2637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:25.680358 2637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:25.680374 2637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:25.685420 2637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:46093
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:46093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:25.686643 2637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:25.688150 2637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:25.699158 2642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:25.699308 2637 server_base.cc:1034] running on GCE node
W20260811 22:26:25.699112 2647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:25.699146 2644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:25.700358 2637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:25.701648 2637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:25.702886 2637 hybrid_clock.cc:648] HybridClock initialized: now 1786487185702853 us; error 58 us; skew 500 ppm
I20260811 22:26:25.705328 2637 webserver.cc:460] Webserver started at http://127.24.71.254:43581/ using document root <none> and password file <none>
I20260811 22:26:25.706073 2637 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:25.706192 2637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:25.706496 2637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:25.708914 2637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "34a3c37664c74e14acac0ae27b3f5586"
format_stamp: "Formatted at 2026-08-11 22:26:25 on dist-test-slave-mvvj"
server_key: "9d077c6a87c33b289730a0cd73b722bd"
server_key_iv: "3530beb773281e92d37eae30a9b8ba26"
server_key_version: "encryptionkey@0"
I20260811 22:26:25.709568 2637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "34a3c37664c74e14acac0ae27b3f5586"
format_stamp: "Formatted at 2026-08-11 22:26:25 on dist-test-slave-mvvj"
server_key: "9d077c6a87c33b289730a0cd73b722bd"
server_key_iv: "3530beb773281e92d37eae30a9b8ba26"
server_key_version: "encryptionkey@0"
I20260811 22:26:25.713711 2637 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:26:25.716837 2653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:25.718338 2637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:25.718551 2637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "34a3c37664c74e14acac0ae27b3f5586"
format_stamp: "Formatted at 2026-08-11 22:26:25 on dist-test-slave-mvvj"
server_key: "9d077c6a87c33b289730a0cd73b722bd"
server_key_iv: "3530beb773281e92d37eae30a9b8ba26"
server_key_version: "encryptionkey@0"
I20260811 22:26:25.718706 2637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:25.737097 2637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:25.737908 2637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:25.738137 2637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:25.746188 2637 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:46093
I20260811 22:26:25.746193 2708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:46093 every 8 connection(s)
I20260811 22:26:25.747227 2637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:26:25.747592 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 2637
I20260811 22:26:25.747720 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:26:25.748139 24863 external_mini_cluster.cc:1428] Setting key b72d5640ade91102bd1a8ae7599d0897
I20260811 22:26:25.751173 2709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:25.758028 2709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586: Bootstrap starting.
I20260811 22:26:25.758452 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:25.749728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56978 (local address 127.24.71.254:46093)
0811 22:26:25.750425 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:25.750443 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:25.750480 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:26:25.751163 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:25.751199 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:25.751651 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:25.752050 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:25.752402 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.753356 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.754621 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:25.755085 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:25.755210 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:25.756493 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:25.756498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:25.756513 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:25.756926 (+ 413us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:25.756988 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:25.757010 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:25.757101 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:25.757280 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 22:26:25.757603 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":159,"threads_started":1}
I20260811 22:26:25.760727 2709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:25.761525 2709 log.cc:824] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:25.763522 2709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586: No bootstrap required, opened a new log
I20260811 22:26:25.767378 2709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a3c37664c74e14acac0ae27b3f5586" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46093 } }
I20260811 22:26:25.767613 2709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:25.767696 2709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a3c37664c74e14acac0ae27b3f5586, State: Initialized, Role: FOLLOWER
I20260811 22:26:25.768162 2709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a3c37664c74e14acac0ae27b3f5586" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46093 } }
I20260811 22:26:25.768287 2709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:26:25.768369 2709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:26:25.768474 2709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:25.769503 2709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a3c37664c74e14acac0ae27b3f5586" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46093 } }
I20260811 22:26:25.769846 2709 leader_election.cc:302] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34a3c37664c74e14acac0ae27b3f5586; no voters:
I20260811 22:26:25.770181 2709 leader_election.cc:288] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:26:25.770300 2715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:25.770619 2715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [term 1 LEADER]: Becoming Leader. State: Replica: 34a3c37664c74e14acac0ae27b3f5586, State: Running, Role: LEADER
I20260811 22:26:25.770959 2715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a3c37664c74e14acac0ae27b3f5586" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46093 } }
I20260811 22:26:25.771523 2709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:25.772936 2717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34a3c37664c74e14acac0ae27b3f5586. Latest consensus state: current_term: 1 leader_uuid: "34a3c37664c74e14acac0ae27b3f5586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a3c37664c74e14acac0ae27b3f5586" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46093 } } }
I20260811 22:26:25.772902 2716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34a3c37664c74e14acac0ae27b3f5586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a3c37664c74e14acac0ae27b3f5586" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46093 } } }
I20260811 22:26:25.773054 2717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:25.773054 2716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586 [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:25.773623 2726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:25.776612 2726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:25.782424 2726 catalog_manager.cc:1347] Generated new cluster ID: 74aafb207ea84774a9e53a5ebca8f5f7
I20260811 22:26:25.782505 2726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:25.800850 2726 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:26:25.801923 2726 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:25.807503 2726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34a3c37664c74e14acac0ae27b3f5586: Generated new TSK 0
I20260811 22:26:25.808427 2726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:26:25.821376 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46093
--builtin_ntp_servers=127.24.71.212:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 22:26:25.949414 2735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:25.949749 2735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:25.949824 2735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:25.953990 2735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:25.954077 2735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:25.954169 2735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:26:25.959067 2735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:25.960366 2735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:25.961897 2735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:25.972515 2743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:25.972522 2740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:25.972525 2741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:25.973061 2735 server_base.cc:1034] running on GCE node
I20260811 22:26:25.973529 2735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:25.974261 2735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:25.975473 2735 hybrid_clock.cc:648] HybridClock initialized: now 1786487185975451 us; error 33 us; skew 500 ppm
I20260811 22:26:25.977660 2735 webserver.cc:460] Webserver started at http://127.24.71.193:32981/ using document root <none> and password file <none>
I20260811 22:26:25.978315 2735 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:25.978417 2735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:25.978678 2735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:25.980962 2735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "c4539a156ef54fe294482fd05b7c0fbc"
format_stamp: "Formatted at 2026-08-11 22:26:25 on dist-test-slave-mvvj"
server_key: "e2e6f2f8d0c2f8a9a41b5de75bbefcbf"
server_key_iv: "892d6e71099223d849440faa31ceefb4"
server_key_version: "encryptionkey@0"
I20260811 22:26:25.981593 2735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "c4539a156ef54fe294482fd05b7c0fbc"
format_stamp: "Formatted at 2026-08-11 22:26:25 on dist-test-slave-mvvj"
server_key: "e2e6f2f8d0c2f8a9a41b5de75bbefcbf"
server_key_iv: "892d6e71099223d849440faa31ceefb4"
server_key_version: "encryptionkey@0"
I20260811 22:26:25.985636 2735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:26:25.988447 2749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:25.989919 2735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:25.990078 2735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "c4539a156ef54fe294482fd05b7c0fbc"
format_stamp: "Formatted at 2026-08-11 22:26:25 on dist-test-slave-mvvj"
server_key: "e2e6f2f8d0c2f8a9a41b5de75bbefcbf"
server_key_iv: "892d6e71099223d849440faa31ceefb4"
server_key_version: "encryptionkey@0"
I20260811 22:26:25.990211 2735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:26.005728 2735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:26.006549 2735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:26.006779 2735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:26.007448 2735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:26.008538 2735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:26.008612 2735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.008688 2735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:26.008739 2735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.018782 2735 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:37731
I20260811 22:26:26.018812 2871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:37731 every 8 connection(s)
I20260811 22:26:26.019915 2735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:26:26.030050 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 2735
I20260811 22:26:26.030128 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:26.022089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:55383 (local address 127.24.71.254:46093)
0811 22:26:26.022260 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:26.022267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:26.023478 (+ 1211us) server_negotiation.cc:407] Connection header received
0811 22:26:26.025076 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:26.025079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:26.025146 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:26.025247 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:26.026214 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.026770 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.028033 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.028262 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.028369 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:26.029567 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:26.029571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:26.029573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:26.029587 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:26.029609 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:26.029617 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:26.029699 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:26.029873 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:26:26.029965 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:26:26.030232 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:26:26.030550 24863 external_mini_cluster.cc:1428] Setting key c8ccd8d2fae8d2838e3177cd7194d695
I20260811 22:26:26.031104 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:26.022510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:46093 (local address 127.24.71.193:55383)
0811 22:26:26.023269 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:26.023364 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:26.024836 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:26.025440 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:26.025447 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:26.025535 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:26.026055 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:26.026069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.026909 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.026919 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:26.027882 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:26.027893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.028435 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.028441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:26.028622 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:26.029273 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:26:26.029274 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:26.029320 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:26:26.029391 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:26.029702 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:26.029715 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:26.029722 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:26:26.029861 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 22:26:26.030180 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":131,"threads_started":1}
I20260811 22:26:26.032549 2872 heartbeater.cc:347] Connected to a master server at 127.24.71.254:46093
I20260811 22:26:26.032812 2872 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:26.032877 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46093
--builtin_ntp_servers=127.24.71.212:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:26:26.033463 2872 heartbeater.cc:511] Master 127.24.71.254:46093 requested a full tablet report, sending...
I20260811 22:26:26.035187 2670 ts_manager.cc:194] Registered new tserver with Master: c4539a156ef54fe294482fd05b7c0fbc (127.24.71.193:37731)
I20260811 22:26:26.036453 2670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.193:55383
W20260811 22:26:26.150322 2875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:26.150624 2875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:26.150662 2875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:26.154874 2875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:26.154955 2875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:26.155012 2875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:26:26.160053 2875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:26.161332 2875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:26.162882 2875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:26.172878 2881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:26.172878 2880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:26.172943 2884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:26.173422 2875 server_base.cc:1034] running on GCE node
I20260811 22:26:26.173840 2875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:26.174557 2875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:26.175725 2875 hybrid_clock.cc:648] HybridClock initialized: now 1786487186175709 us; error 30 us; skew 500 ppm
I20260811 22:26:26.177696 2875 webserver.cc:460] Webserver started at http://127.24.71.194:46155/ using document root <none> and password file <none>
I20260811 22:26:26.178344 2875 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:26.178414 2875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:26.178655 2875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:26.180879 2875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "74f3eb897feb46dda67bc9524f99a932"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "5c4cc677ef41cd911b53f51715e45174"
server_key_iv: "2281bff0bf264d1047e390d0662b7a5b"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.181471 2875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "74f3eb897feb46dda67bc9524f99a932"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "5c4cc677ef41cd911b53f51715e45174"
server_key_iv: "2281bff0bf264d1047e390d0662b7a5b"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.185283 2875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:26:26.188033 2890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.189319 2875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:26:26.189487 2875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "74f3eb897feb46dda67bc9524f99a932"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "5c4cc677ef41cd911b53f51715e45174"
server_key_iv: "2281bff0bf264d1047e390d0662b7a5b"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.189617 2875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:26.203512 2875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:26.204380 2875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:26.204589 2875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:26.205195 2875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:26.206353 2875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:26.206418 2875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.206490 2875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:26.206525 2875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.215894 2875 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:45933
I20260811 22:26:26.215934 3009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:45933 every 8 connection(s)
I20260811 22:26:26.216995 2875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:26:26.220871 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 2875
I20260811 22:26:26.221006 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:26:26.221340 24863 external_mini_cluster.cc:1428] Setting key 7666ec5dc56be7bb3179df3d3fce7b5e
I20260811 22:26:26.224089 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46093
--builtin_ntp_servers=127.24.71.212:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:26:26.228725 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:26.219071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:41627 (local address 127.24.71.254:46093)
0811 22:26:26.219231 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:26.219235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:26.220461 (+ 1226us) server_negotiation.cc:407] Connection header received
0811 22:26:26.222513 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:26.222516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:26.222571 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:26.222636 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:26.223771 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.224420 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.226239 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.226641 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.226780 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:26.228088 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:26.228095 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:26.228101 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:26.228121 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:26.228150 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:26.228163 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:26.228247 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:26.228397 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 22:26:26.228475 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:26:26.229697 3012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:26.219495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:46093 (local address 127.24.71.194:41627)
0811 22:26:26.220248 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:26.220292 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:26.222216 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:26.222836 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:26.222845 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:26.222949 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:26.223612 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:26.223634 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.224553 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.224560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:26.226085 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:26.226097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.226832 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.226836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:26.227016 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:26.227745 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:26:26.227747 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:26.227791 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:26:26.227919 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:26.228254 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:26.228262 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:26.228267 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:26:26.228370 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:26:26.228683 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":134,"threads_started":1}
I20260811 22:26:26.230922 3010 heartbeater.cc:347] Connected to a master server at 127.24.71.254:46093
I20260811 22:26:26.231184 3010 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:26.231760 3010 heartbeater.cc:511] Master 127.24.71.254:46093 requested a full tablet report, sending...
I20260811 22:26:26.232933 2670 ts_manager.cc:194] Registered new tserver with Master: 74f3eb897feb46dda67bc9524f99a932 (127.24.71.194:45933)
I20260811 22:26:26.233547 2670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.194:41627
W20260811 22:26:26.349902 3013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:26.350193 3013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:26.350232 3013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:26.354455 3013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:26.354527 3013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:26.354586 3013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:26:26.359539 3013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:26.360800 3013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:26.362450 3013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:26.373414 3019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:26.373442 3021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:26.373428 3018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:26.373474 3013 server_base.cc:1034] running on GCE node
I20260811 22:26:26.374178 3013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:26.374874 3013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:26.376037 3013 hybrid_clock.cc:648] HybridClock initialized: now 1786487186376016 us; error 33 us; skew 500 ppm
I20260811 22:26:26.377947 3013 webserver.cc:460] Webserver started at http://127.24.71.195:34401/ using document root <none> and password file <none>
I20260811 22:26:26.378587 3013 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:26.378655 3013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:26.378829 3013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:26.381049 3013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "4209b5f5fecf492b97e953de6a8c5ec1"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "942d8799b284ec5fc467ceb8bfadca35"
server_key_iv: "2f406ea6ae2f1c8bd7b468e30f153d61"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.381671 3013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "4209b5f5fecf492b97e953de6a8c5ec1"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "942d8799b284ec5fc467ceb8bfadca35"
server_key_iv: "2f406ea6ae2f1c8bd7b468e30f153d61"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.385425 3013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:26:26.388026 3028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.389302 3013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:26:26.389410 3013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "4209b5f5fecf492b97e953de6a8c5ec1"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "942d8799b284ec5fc467ceb8bfadca35"
server_key_iv: "2f406ea6ae2f1c8bd7b468e30f153d61"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.389560 3013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:26.412838 3013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:26.413652 3013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:26.413873 3013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:26.414535 3013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:26.415593 3013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:26.415643 3013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.415714 3013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:26.415750 3013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.425452 3013 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:36159
I20260811 22:26:26.425480 3145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:36159 every 8 connection(s)
I20260811 22:26:26.426577 3013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:26:26.433903 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 3013
I20260811 22:26:26.434074 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:26:26.434458 24863 external_mini_cluster.cc:1428] Setting key be07adb398aec675ee4de4929587e01f
I20260811 22:26:26.437182 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:26.428913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:56841 (local address 127.24.71.254:46093)
0811 22:26:26.429116 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:26.429121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:26.430295 (+ 1174us) server_negotiation.cc:407] Connection header received
0811 22:26:26.431771 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:26.431774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:26.431878 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:26.431971 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:26.433033 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.433598 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.435067 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.435295 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.435410 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:26.436644 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:26.436648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:26.436650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:26.436662 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:26.436681 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:26.436689 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:26.436769 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:26.436945 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:26:26.437013 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:26:26.438169 3148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:26.429353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:46093 (local address 127.24.71.195:56841)
0811 22:26:26.430131 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:26.430175 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:26.431499 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:26.432172 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:26.432179 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:26.432265 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:26.432867 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:26.432882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.433784 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.433788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:26.434943 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:26.434958 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:26.435458 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:26.435463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:26.435644 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:26.436353 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:26:26.436354 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:26.436398 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:26:26.436469 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:26.436772 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:26.436785 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:26.436792 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:26:26.436934 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 22:26:26.437249 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":126,"threads_started":1}
I20260811 22:26:26.439381 3146 heartbeater.cc:347] Connected to a master server at 127.24.71.254:46093
I20260811 22:26:26.439692 3146 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:26.440341 3146 heartbeater.cc:511] Master 127.24.71.254:46093 requested a full tablet report, sending...
I20260811 22:26:26.441434 2670 ts_manager.cc:194] Registered new tserver with Master: 4209b5f5fecf492b97e953de6a8c5ec1 (127.24.71.195:36159)
I20260811 22:26:26.442037 2670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.195:56841
I20260811 22:26:26.449067 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:26:26.452138 2713 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:26.451118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.24.71.254:46093)
0811 22:26:26.451293 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:26.451297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:26.451391 (+ 94us) server_negotiation.cc:407] Connection header received
0811 22:26:26.451495 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:26.451497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:26.451541 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:26.451610 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:26.452008 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43302: BlockingRecv error: recv got EOF from 127.0.0.1:43302 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:26:26.452535 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 2735
I20260811 22:26:26.464497 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 2875
I20260811 22:26:26.472867 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 3013
I20260811 22:26:26.481388 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 2637
2026-08-11T22:26:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (967 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T22:26:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:26Z Disabled control of system clock
I20260811 22:26:26.517513 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38967
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:38967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:26:26.645846 3157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:26.646160 3157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:26.646200 3157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:26.650409 3157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:26.650487 3157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:26.650503 3157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:26.650528 3157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:26.650544 3157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:26.656562 3157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:38967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:38967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:26.657768 3157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:26.659260 3157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:26.668632 3166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:26.668613 3164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:26.668726 3157 server_base.cc:1034] running on GCE node
W20260811 22:26:26.668718 3162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:26.669656 3157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:26.670812 3157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:26.671979 3157 hybrid_clock.cc:648] HybridClock initialized: now 1786487186671960 us; error 32 us; skew 500 ppm
I20260811 22:26:26.673931 3157 webserver.cc:460] Webserver started at http://127.24.71.254:46371/ using document root <none> and password file <none>
I20260811 22:26:26.674503 3157 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:26.674569 3157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:26.674741 3157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:26.676915 3157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "75a275ff72bf4e648273af7f6b27fa5f"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "6e9e571f95e1ed62b364672bb6d24dc2"
server_key_iv: "c90c53719dd915f672cdb5f6b86d38e0"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.677457 3157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "75a275ff72bf4e648273af7f6b27fa5f"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "6e9e571f95e1ed62b364672bb6d24dc2"
server_key_iv: "c90c53719dd915f672cdb5f6b86d38e0"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.681339 3157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:26:26.683971 3173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.685422 3157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:26.685560 3157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "75a275ff72bf4e648273af7f6b27fa5f"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "6e9e571f95e1ed62b364672bb6d24dc2"
server_key_iv: "c90c53719dd915f672cdb5f6b86d38e0"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.685681 3157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:26.690436 3157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:26.691069 3157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:26.691256 3157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:26.699471 3157 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:38967
I20260811 22:26:26.699478 3229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:38967 every 8 connection(s)
I20260811 22:26:26.700565 3157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:26:26.704191 3230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:26.704487 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 3157
I20260811 22:26:26.704653 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:26:26.704967 24863 external_mini_cluster.cc:1428] Setting key 44b47d35bfcbc748994e4d019cf867e8
I20260811 22:26:26.710950 3230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f: Bootstrap starting.
I20260811 22:26:26.711247 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:26.706388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.24.71.254:38967)
0811 22:26:26.707035 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:26.707053 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:26.707444 (+ 391us) server_negotiation.cc:407] Connection header received
0811 22:26:26.708319 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:26.708339 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:26.708711 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:26.708995 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:26.709878 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:26.709883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:26.709896 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:26.709906 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:26.709964 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:26.709979 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:26.710130 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:26.710188 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 22:26:26.710467 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":138,"threads_started":1}
I20260811 22:26:26.713845 3230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:26.714619 3230 log.cc:824] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:26.717275 3230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f: No bootstrap required, opened a new log
I20260811 22:26:26.721311 3230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a275ff72bf4e648273af7f6b27fa5f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38967 } }
I20260811 22:26:26.721587 3230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:26.721670 3230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a275ff72bf4e648273af7f6b27fa5f, State: Initialized, Role: FOLLOWER
I20260811 22:26:26.722122 3230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a275ff72bf4e648273af7f6b27fa5f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38967 } }
I20260811 22:26:26.722278 3230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:26:26.722368 3230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:26:26.722469 3230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:26.723459 3230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a275ff72bf4e648273af7f6b27fa5f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38967 } }
I20260811 22:26:26.723814 3230 leader_election.cc:302] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75a275ff72bf4e648273af7f6b27fa5f; no voters:
I20260811 22:26:26.724229 3230 leader_election.cc:288] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:26:26.724431 3235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:26.724767 3235 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [term 1 LEADER]: Becoming Leader. State: Replica: 75a275ff72bf4e648273af7f6b27fa5f, State: Running, Role: LEADER
I20260811 22:26:26.725139 3235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a275ff72bf4e648273af7f6b27fa5f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38967 } }
I20260811 22:26:26.725608 3230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:26.726933 3238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75a275ff72bf4e648273af7f6b27fa5f. Latest consensus state: current_term: 1 leader_uuid: "75a275ff72bf4e648273af7f6b27fa5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a275ff72bf4e648273af7f6b27fa5f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38967 } } }
I20260811 22:26:26.727085 3238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:26.727336 3236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75a275ff72bf4e648273af7f6b27fa5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a275ff72bf4e648273af7f6b27fa5f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 38967 } } }
I20260811 22:26:26.727512 3236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f [sys.catalog]: This master's current role is: LEADER
W20260811 22:26:26.730667 3251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:26:26.730772 3251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:26:26.730846 3245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:26.731598 3245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:26.737601 3245 catalog_manager.cc:1347] Generated new cluster ID: ecc5d734e6f24e178203b7a745d78112
I20260811 22:26:26.737685 3245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:26.761461 3245 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:26:26.761868 3245 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:26.767452 3245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75a275ff72bf4e648273af7f6b27fa5f: Generated new TSK 0
I20260811 22:26:26.768350 3245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:26:26.781971 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38967
--builtin_ntp_servers=127.24.71.212:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:26:26.903635 3255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:26.903987 3255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:26.904035 3255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:26.908246 3255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:26.908304 3255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:26.908388 3255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:26:26.913365 3255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:26.914599 3255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:26.916105 3255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:26.926332 3262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:26.926332 3260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:26.926586 3255 server_base.cc:1034] running on GCE node
W20260811 22:26:26.926458 3264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:26.927142 3255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:26.927951 3255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:26.929139 3255 hybrid_clock.cc:648] HybridClock initialized: now 1786487186929120 us; error 29 us; skew 500 ppm
I20260811 22:26:26.931180 3255 webserver.cc:460] Webserver started at http://127.24.71.193:34777/ using document root <none> and password file <none>
I20260811 22:26:26.931782 3255 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:26.931908 3255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:26.932145 3255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:26.934262 3255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "b68eaad11b11415e87ea8db5418fd711"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "83d44b83459efa0c5c93b3bea2895c4f"
server_key_iv: "4d73110e2a7052954039c45abab821c5"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.934849 3255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "b68eaad11b11415e87ea8db5418fd711"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "83d44b83459efa0c5c93b3bea2895c4f"
server_key_iv: "4d73110e2a7052954039c45abab821c5"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.938701 3255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 22:26:26.941471 3270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.942747 3255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:26:26.942888 3255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "b68eaad11b11415e87ea8db5418fd711"
format_stamp: "Formatted at 2026-08-11 22:26:26 on dist-test-slave-mvvj"
server_key: "83d44b83459efa0c5c93b3bea2895c4f"
server_key_iv: "4d73110e2a7052954039c45abab821c5"
server_key_version: "encryptionkey@0"
I20260811 22:26:26.943017 3255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:26.947685 3255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:26.948379 3255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:26.948570 3255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:26.949131 3255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:26.950117 3255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:26.950191 3255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.950259 3255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:26.950309 3255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:26.960137 3255 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:45917
I20260811 22:26:26.960208 3388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:45917 every 8 connection(s)
I20260811 22:26:26.961140 3255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:26:26.967571 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:26.963080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:60277 (local address 127.24.71.254:38967)
0811 22:26:26.963221 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:26.963224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:26.964448 (+ 1224us) server_negotiation.cc:407] Connection header received
0811 22:26:26.965845 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:26.965848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:26.965892 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:26.966016 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:26.967200 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:26.967203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:26.967206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:26.967217 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:26.967236 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:26.967243 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:26.967324 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:26.967412 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 22:26:26.967451 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:26:26.968554 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:26.963526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38967 (local address 127.24.71.193:60277)
0811 22:26:26.964289 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:26.964332 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:26.965607 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:26.966189 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:26.966196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:26.966292 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:26.966932 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:26:26.966934 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:26.966980 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:26:26.967053 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:26.967305 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:26.967310 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:26.967313 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 22:26:26.967439 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 22:26:26.967683 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":121,"threads_started":1}
I20260811 22:26:26.969884 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 3255
I20260811 22:26:26.970018 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:26:26.970031 3389 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38967
I20260811 22:26:26.970284 3389 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:26.970395 24863 external_mini_cluster.cc:1428] Setting key a9fe61a96fb4d02676b9999488a37665
I20260811 22:26:26.970801 3389 heartbeater.cc:511] Master 127.24.71.254:38967 requested a full tablet report, sending...
I20260811 22:26:26.972625 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38967
--builtin_ntp_servers=127.24.71.212:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:26:26.972640 3186 ts_manager.cc:194] Registered new tserver with Master: b68eaad11b11415e87ea8db5418fd711 (127.24.71.193:45917)
W20260811 22:26:27.093086 3392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:27.093376 3392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:27.093420 3392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:27.097587 3392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:27.097661 3392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:27.097720 3392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:26:27.102692 3392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:27.103881 3392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:27.105352 3392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:27.115983 3401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:27.116227 3397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:27.116248 3399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:27.116410 3392 server_base.cc:1034] running on GCE node
I20260811 22:26:27.116861 3392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:27.117609 3392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:27.118810 3392 hybrid_clock.cc:648] HybridClock initialized: now 1786487187118746 us; error 78 us; skew 500 ppm
I20260811 22:26:27.120841 3392 webserver.cc:460] Webserver started at http://127.24.71.194:42819/ using document root <none> and password file <none>
I20260811 22:26:27.121457 3392 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:27.121556 3392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:27.121800 3392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:27.124078 3392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "c863efa4b3f84ba3b998ff78684be260"
format_stamp: "Formatted at 2026-08-11 22:26:27 on dist-test-slave-mvvj"
server_key: "821734ead61e729ddd76e984d49ca0eb"
server_key_iv: "9e53b99a8d297591e11bd147786cafbe"
server_key_version: "encryptionkey@0"
I20260811 22:26:27.124701 3392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "c863efa4b3f84ba3b998ff78684be260"
format_stamp: "Formatted at 2026-08-11 22:26:27 on dist-test-slave-mvvj"
server_key: "821734ead61e729ddd76e984d49ca0eb"
server_key_iv: "9e53b99a8d297591e11bd147786cafbe"
server_key_version: "encryptionkey@0"
I20260811 22:26:27.128887 3392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:26:27.131620 3408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:27.132984 3392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:27.133145 3392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "c863efa4b3f84ba3b998ff78684be260"
format_stamp: "Formatted at 2026-08-11 22:26:27 on dist-test-slave-mvvj"
server_key: "821734ead61e729ddd76e984d49ca0eb"
server_key_iv: "9e53b99a8d297591e11bd147786cafbe"
server_key_version: "encryptionkey@0"
I20260811 22:26:27.133277 3392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:27.138008 3392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:27.138643 3392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:27.138832 3392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:27.139390 3392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:27.140426 3392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:27.140496 3392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:27.140568 3392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:27.140622 3392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:27.149629 3392 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:34563
I20260811 22:26:27.149654 3528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:34563 every 8 connection(s)
I20260811 22:26:27.150532 3392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:26:27.156780 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:27.152401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:50383 (local address 127.24.71.254:38967)
0811 22:26:27.152589 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:27.152594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:27.153659 (+ 1065us) server_negotiation.cc:407] Connection header received
0811 22:26:27.154956 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:27.154960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:27.155002 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:27.155085 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:27.156287 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:27.156295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:27.156297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:27.156310 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:27.156384 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:27.156393 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:27.156479 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:27.156625 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 22:26:27.156668 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:26:27.157691 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:27.152802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38967 (local address 127.24.71.194:50383)
0811 22:26:27.153485 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:27.153528 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:27.154729 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:27.155257 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:27.155264 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:27.155354 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:27.156012 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:26:27.156014 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:27.156055 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:26:27.156125 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:27.156485 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:27.156493 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:27.156497 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:26:27.156634 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 22:26:27.156857 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":127,"threads_started":1}
I20260811 22:26:27.159008 3529 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38967
I20260811 22:26:27.159278 3529 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:27.159721 3529 heartbeater.cc:511] Master 127.24.71.254:38967 requested a full tablet report, sending...
I20260811 22:26:27.160715 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 3392
I20260811 22:26:27.160862 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:26:27.161017 3186 ts_manager.cc:194] Registered new tserver with Master: c863efa4b3f84ba3b998ff78684be260 (127.24.71.194:34563)
I20260811 22:26:27.161232 24863 external_mini_cluster.cc:1428] Setting key a83d1ec0fc3458b7f75cc3aefeb68ac1
I20260811 22:26:27.163302 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38967
--builtin_ntp_servers=127.24.71.212:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:26:27.281558 3532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:27.281855 3532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:27.281898 3532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:27.286031 3532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:27.286095 3532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:27.286156 3532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:26:27.291253 3532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:38967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:27.292492 3532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:27.293885 3532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:27.304461 3541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:27.304482 3537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:27.304473 3538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:27.304543 3532 server_base.cc:1034] running on GCE node
I20260811 22:26:27.305147 3532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:27.305817 3532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:27.307001 3532 hybrid_clock.cc:648] HybridClock initialized: now 1786487187306964 us; error 48 us; skew 500 ppm
I20260811 22:26:27.309155 3532 webserver.cc:460] Webserver started at http://127.24.71.195:36881/ using document root <none> and password file <none>
I20260811 22:26:27.309755 3532 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:27.309859 3532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:27.310089 3532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:27.312314 3532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "6a04f16404324a2eb346cabcab4064d4"
format_stamp: "Formatted at 2026-08-11 22:26:27 on dist-test-slave-mvvj"
server_key: "bb7be19abb72232f0d5635119757687e"
server_key_iv: "3aa68e075f40725f344bcc587425bdf4"
server_key_version: "encryptionkey@0"
I20260811 22:26:27.312855 3532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "6a04f16404324a2eb346cabcab4064d4"
format_stamp: "Formatted at 2026-08-11 22:26:27 on dist-test-slave-mvvj"
server_key: "bb7be19abb72232f0d5635119757687e"
server_key_iv: "3aa68e075f40725f344bcc587425bdf4"
server_key_version: "encryptionkey@0"
I20260811 22:26:27.317015 3532 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:26:27.320086 3547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:27.321514 3532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:27.321674 3532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "6a04f16404324a2eb346cabcab4064d4"
format_stamp: "Formatted at 2026-08-11 22:26:27 on dist-test-slave-mvvj"
server_key: "bb7be19abb72232f0d5635119757687e"
server_key_iv: "3aa68e075f40725f344bcc587425bdf4"
server_key_version: "encryptionkey@0"
I20260811 22:26:27.321810 3532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:27.326772 3532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:27.327436 3532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:27.327639 3532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:27.328303 3532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:27.329276 3532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:27.329346 3532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:27.329411 3532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:27.329468 3532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:27.339416 3532 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:33289
I20260811 22:26:27.339450 3668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:33289 every 8 connection(s)
I20260811 22:26:27.340478 3532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:26:27.346977 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:27.342498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:54981 (local address 127.24.71.254:38967)
0811 22:26:27.342657 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:27.342661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:27.343810 (+ 1149us) server_negotiation.cc:407] Connection header received
0811 22:26:27.345202 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:27.345206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:27.345254 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:27.345376 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:27.346504 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:27.346508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:27.346510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:26:27.346522 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:27.346543 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:26:27.346552 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:27.346635 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:27.346819 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 22:26:27.346866 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:26:27.347982 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:27.342900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:38967 (local address 127.24.71.195:54981)
0811 22:26:27.343633 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:27.343677 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:27.344979 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:27.345541 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:27.345548 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:27.345635 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:27.346238 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:26:27.346239 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:27.346280 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:26:27.346350 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:27.346671 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:27.346679 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:27.346685 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:26:27.346839 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 22:26:27.347060 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":123,"threads_started":1}
I20260811 22:26:27.349262 3669 heartbeater.cc:347] Connected to a master server at 127.24.71.254:38967
I20260811 22:26:27.349476 3669 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:27.349877 3669 heartbeater.cc:511] Master 127.24.71.254:38967 requested a full tablet report, sending...
I20260811 22:26:27.350819 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 3532
I20260811 22:26:27.350955 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:26:27.351012 3186 ts_manager.cc:194] Registered new tserver with Master: 6a04f16404324a2eb346cabcab4064d4 (127.24.71.195:33289)
I20260811 22:26:27.351331 24863 external_mini_cluster.cc:1428] Setting key 9151cbb091580905277c1f3bbd7d4254
I20260811 22:26:27.353709 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:26:27.356707 3233 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:26:27.355959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51340 (local address 127.24.71.254:38967)
0811 22:26:27.356098 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:27.356102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:27.356135 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:26:27.356242 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:27.356245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:27.356275 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:27.356360 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:27.356555 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51340: BlockingRecv error: recv got EOF from 127.0.0.1:51340 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:26:27.357082 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 3255
I20260811 22:26:27.370034 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 3392
I20260811 22:26:27.381649 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 3532
I20260811 22:26:27.389541 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 3157
2026-08-11T22:26:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (907 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:26:27 dist-test-slave-mvvj krb5kdc[3679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:26:27 dist-test-slave-mvvj krb5kdc[3679](info): set up 2 sockets
Aug 11 22:26:27 dist-test-slave-mvvj krb5kdc[3679](info): commencing operation
krb5kdc: starting...
W20260811 22:26:28.544560 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:26:28 dist-test-slave-mvvj krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:26:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:26:28.679404 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42449
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:42449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:28.797472 3693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:28.797804 3693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:28.797892 3693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:28.802037 3693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:26:28.802137 3693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:28.802178 3693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:28.802218 3693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:26:28.802253 3693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:26:28.807821 3693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:42449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:28.809033 3693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:28.810125 3693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:26:28.820070 3693 server_base.cc:1034] running on GCE node
W20260811 22:26:28.820015 3699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:28.820011 3703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:28.820011 3700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:28.820932 3693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:28.822142 3693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:28.823354 3693 hybrid_clock.cc:648] HybridClock initialized: now 1786487188823328 us; error 34 us; skew 500 ppm
Aug 11 22:26:28 dist-test-slave-mvvj krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487188, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:28.826328 3693 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:26:28.827302 3693 webserver.cc:460] Webserver started at http://127.24.71.254:44155/ using document root <none> and password file <none>
I20260811 22:26:28.827960 3693 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:28.828058 3693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:28.828306 3693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:28.830423 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "13328d08e633435db5dd90667e57d2ac"
format_stamp: "Formatted at 2026-08-11 22:26:28 on dist-test-slave-mvvj"
server_key: "3bf6fbd4e73af782789ef0bccfc8783b"
server_key_iv: "e7e6411d87d1968f4613ad1ca8b5b4b1"
server_key_version: "encryptionkey@0"
I20260811 22:26:28.831025 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "13328d08e633435db5dd90667e57d2ac"
format_stamp: "Formatted at 2026-08-11 22:26:28 on dist-test-slave-mvvj"
server_key: "3bf6fbd4e73af782789ef0bccfc8783b"
server_key_iv: "e7e6411d87d1968f4613ad1ca8b5b4b1"
server_key_version: "encryptionkey@0"
I20260811 22:26:28.834903 3693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:26:28.837648 3710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:28.839097 3693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:28.839250 3693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "13328d08e633435db5dd90667e57d2ac"
format_stamp: "Formatted at 2026-08-11 22:26:28 on dist-test-slave-mvvj"
server_key: "3bf6fbd4e73af782789ef0bccfc8783b"
server_key_iv: "e7e6411d87d1968f4613ad1ca8b5b4b1"
server_key_version: "encryptionkey@0"
I20260811 22:26:28.839381 3693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:28.889253 3693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:28.892588 3693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:28.892817 3693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:28.901168 3693 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:42449
I20260811 22:26:28.901217 3764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:42449 every 8 connection(s)
I20260811 22:26:28.902269 3693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:26:28.905472 3765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:28.908265 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 3693
I20260811 22:26:28.908412 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:26:28.908733 24863 external_mini_cluster.cc:1428] Setting key 11dcd1fecd10dda852b4da96e5e25211
I20260811 22:26:28.912333 3765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac: Bootstrap starting.
I20260811 22:26:28.915251 3765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:28.916211 3765 log.cc:824] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:26:28 dist-test-slave-mvvj krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:26:28.918521 3765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac: No bootstrap required, opened a new log
I20260811 22:26:28.923275 3765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13328d08e633435db5dd90667e57d2ac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42449 } }
I20260811 22:26:28.923596 3765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:28.923655 3765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13328d08e633435db5dd90667e57d2ac, State: Initialized, Role: FOLLOWER
I20260811 22:26:28.924221 3765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13328d08e633435db5dd90667e57d2ac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42449 } }
I20260811 22:26:28.924394 3765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:26:28.924501 3765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:26:28.924614 3765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:28.925737 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:28.910342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.24.71.254:42449)
0811 22:26:28.910814 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:28.910832 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:28.910873 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:26:28.911535 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:28.911554 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:28.912004 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:28.912308 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:28.913270 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:28.914351 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:28.915539 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:28.916123 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:28.916250 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:28.918923 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:28.918933 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:28.918952 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:28.918987 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:28.922046 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:28.922623 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:28.922630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:28.922637 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:28.922789 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:28.923093 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:28.923096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:28.923098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:28.923672 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:28.923908 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:28.924321 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 22:26:28.924677 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":147,"threads_started":1}
I20260811 22:26:28.925908 3765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13328d08e633435db5dd90667e57d2ac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42449 } }
I20260811 22:26:28.926301 3765 leader_election.cc:302] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13328d08e633435db5dd90667e57d2ac; no voters:
I20260811 22:26:28.926682 3765 leader_election.cc:288] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:26:28.926831 3772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:28.927080 3772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [term 1 LEADER]: Becoming Leader. State: Replica: 13328d08e633435db5dd90667e57d2ac, State: Running, Role: LEADER
I20260811 22:26:28.927443 3772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13328d08e633435db5dd90667e57d2ac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42449 } }
I20260811 22:26:28.928071 3765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:26:28.928861 3773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13328d08e633435db5dd90667e57d2ac. Latest consensus state: current_term: 1 leader_uuid: "13328d08e633435db5dd90667e57d2ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13328d08e633435db5dd90667e57d2ac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42449 } } }
I20260811 22:26:28.929040 3773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:28.929507 3779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:26:28.929428 3774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13328d08e633435db5dd90667e57d2ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13328d08e633435db5dd90667e57d2ac" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42449 } } }
I20260811 22:26:28.929625 3774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac [sys.catalog]: This master's current role is: LEADER
I20260811 22:26:28.932844 3779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:26:28.939194 3779 catalog_manager.cc:1347] Generated new cluster ID: dbe95b57780241638a8178db3f9080e3
I20260811 22:26:28.939301 3779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:26:28.948966 3779 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:26:28.950014 3779 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:26:28.957139 3779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac: Generated new TSK 0
I20260811 22:26:28.957955 3779 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:26:28.995040 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42449
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:29.117062 3794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:29.117421 3794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:29.117501 3794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:29.121731 3794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:29.121805 3794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:29.121898 3794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:26:29.127490 3794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:29.128752 3794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:29.129920 3794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:29.140094 3802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:29.140115 3799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:29.140097 3800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:29.140138 3794 server_base.cc:1034] running on GCE node
I20260811 22:26:29.140833 3794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:29.141512 3794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:29.142740 3794 hybrid_clock.cc:648] HybridClock initialized: now 1786487189142719 us; error 30 us; skew 500 ppm
Aug 11 22:26:29 dist-test-slave-mvvj krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:29.145695 3794 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:26:29.146593 3794 webserver.cc:460] Webserver started at http://127.24.71.193:38905/ using document root <none> and password file <none>
I20260811 22:26:29.147179 3794 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:29.147274 3794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:29.147526 3794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:29.149669 3794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "5b8845c46e4046be9c70fe72f248b76e"
format_stamp: "Formatted at 2026-08-11 22:26:29 on dist-test-slave-mvvj"
server_key: "5fe5f61c579779d4f8e06a631dc29cdd"
server_key_iv: "6f34f3948fa60e47e595275d8775f26a"
server_key_version: "encryptionkey@0"
I20260811 22:26:29.150251 3794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "5b8845c46e4046be9c70fe72f248b76e"
format_stamp: "Formatted at 2026-08-11 22:26:29 on dist-test-slave-mvvj"
server_key: "5fe5f61c579779d4f8e06a631dc29cdd"
server_key_iv: "6f34f3948fa60e47e595275d8775f26a"
server_key_version: "encryptionkey@0"
I20260811 22:26:29.154193 3794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:26:29.156960 3810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:29.158245 3794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:26:29.158412 3794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "5b8845c46e4046be9c70fe72f248b76e"
format_stamp: "Formatted at 2026-08-11 22:26:29 on dist-test-slave-mvvj"
server_key: "5fe5f61c579779d4f8e06a631dc29cdd"
server_key_iv: "6f34f3948fa60e47e595275d8775f26a"
server_key_version: "encryptionkey@0"
I20260811 22:26:29.158567 3794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:29.168545 3794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:29.171584 3794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:29.171794 3794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:29.172417 3794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:29.173218 3794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:29.173264 3794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:29.173305 3794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:29.173321 3794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:29.182823 3794 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:40465
I20260811 22:26:29.182852 3931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:40465 every 8 connection(s)
I20260811 22:26:29.183873 3794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
Aug 11 22:26:29 dist-test-slave-mvvj krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:26:29.194067 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 3794
I20260811 22:26:29.194202 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:26:29.194551 24863 external_mini_cluster.cc:1428] Setting key 75cfdc367dbd53fed2ca404937e8b6f7
I20260811 22:26:29.198933 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.185802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:39417 (local address 127.24.71.254:42449)
0811 22:26:29.185951 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.185966 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.186790 (+ 824us) server_negotiation.cc:407] Connection header received
0811 22:26:29.188338 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.188342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.188404 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.188488 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:29.189697 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.190254 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.191404 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.191664 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.191776 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.194477 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:29.194481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:29.194484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:29.194543 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:29.196366 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:29.197025 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:29.197028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:29.197029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:29.197107 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:29.197538 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:29.197541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:29.197542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:29.197720 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:29.197795 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.198504 (+ 709us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.198709 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:26:29.199695 3934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.186181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42449 (local address 127.24.71.193:39417)
0811 22:26:29.186639 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:29.186683 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:29.188114 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:29.188683 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:29.188690 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:29.189056 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:29.189538 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.189552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.190389 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.190394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.191267 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.191277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.191876 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.191885 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.192048 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.192685 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:29.192702 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:29.194281 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:29.196517 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:29.196523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:29.196539 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:29.196916 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:29.197198 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:29.197204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:29.197209 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:29.197406 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:29.197822 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:29.197832 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:29.198178 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 22:26:29.198441 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 22:26:29.198751 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
I20260811 22:26:29.201054 3932 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42449
I20260811 22:26:29.201354 3932 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:29.201875 3932 heartbeater.cc:511] Master 127.24.71.254:42449 requested a full tablet report, sending...
I20260811 22:26:29.203512 3727 ts_manager.cc:194] Registered new tserver with Master: 5b8845c46e4046be9c70fe72f248b76e (127.24.71.193:40465)
I20260811 22:26:29.204730 3727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:39417
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:26:29.228169 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42449
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:29.347105 3937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:29.347360 3937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:29.347405 3937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:29.351462 3937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:29.351537 3937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:29.351596 3937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:26:29.357095 3937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:29.358311 3937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:29.359418 3937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:26:29.370046 3937 server_base.cc:1034] running on GCE node
W20260811 22:26:29.369976 3943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:29.370024 3945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:29.369992 3942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:29.370632 3937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:29.371390 3937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:29.372553 3937 hybrid_clock.cc:648] HybridClock initialized: now 1786487189372520 us; error 57 us; skew 500 ppm
Aug 11 22:26:29 dist-test-slave-mvvj krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:29.379637 3937 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:26:29.380703 3937 webserver.cc:460] Webserver started at http://127.24.71.194:36363/ using document root <none> and password file <none>
I20260811 22:26:29.381327 3937 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:29.381423 3937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:29.381670 3937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:29.383912 3937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "2383151068aa41549dd7f04bc78f73ac"
format_stamp: "Formatted at 2026-08-11 22:26:29 on dist-test-slave-mvvj"
server_key: "355442a974f15c168190e6742e20f40f"
server_key_iv: "25c2bc73220d6d40807cdf21d6032121"
server_key_version: "encryptionkey@0"
I20260811 22:26:29.384532 3937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "2383151068aa41549dd7f04bc78f73ac"
format_stamp: "Formatted at 2026-08-11 22:26:29 on dist-test-slave-mvvj"
server_key: "355442a974f15c168190e6742e20f40f"
server_key_iv: "25c2bc73220d6d40807cdf21d6032121"
server_key_version: "encryptionkey@0"
I20260811 22:26:29.388562 3937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:26:29.391119 3952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:29.392438 3937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:26:29.392583 3937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "2383151068aa41549dd7f04bc78f73ac"
format_stamp: "Formatted at 2026-08-11 22:26:29 on dist-test-slave-mvvj"
server_key: "355442a974f15c168190e6742e20f40f"
server_key_iv: "25c2bc73220d6d40807cdf21d6032121"
server_key_version: "encryptionkey@0"
I20260811 22:26:29.392705 3937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:29.406659 3937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:29.410079 3937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:29.410291 3937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:29.410887 3937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:29.411736 3937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:29.411808 3937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:29.411906 3937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:29.411952 3937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:29.421562 3937 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:40223
I20260811 22:26:29.421586 4077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:40223 every 8 connection(s)
I20260811 22:26:29.422487 3937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:26:29.426862 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 3937
I20260811 22:26:29.427006 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:26:29.427315 24863 external_mini_cluster.cc:1428] Setting key 1f7e68835edb763cabbacc5e040ade25
Aug 11 22:26:29 dist-test-slave-mvvj krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:26:29.438728 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.424513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:52851 (local address 127.24.71.254:42449)
0811 22:26:29.424657 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.424664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.425490 (+ 826us) server_negotiation.cc:407] Connection header received
0811 22:26:29.426917 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.426921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.426985 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.427069 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:29.428633 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.429401 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.431256 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.431658 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.431807 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.434794 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:29.434798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:29.434800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:29.434831 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:29.436473 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:29.437058 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:29.437062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:29.437064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:29.437152 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:29.437531 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:29.437535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:29.437536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:29.437740 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:29.437854 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.438394 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.438544 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:26:29.439553 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.424864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42449 (local address 127.24.71.194:52851)
0811 22:26:29.425331 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:29.425375 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:29.426696 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:29.427245 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:29.427252 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:29.427703 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:29.428374 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.428389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.429581 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.429586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.431086 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.431102 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.431883 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.431894 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.432057 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.432709 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:29.432727 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:29.434594 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:29.436634 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:29.436640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:29.436659 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:29.436930 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:29.437256 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:29.437259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:29.437261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:29.437421 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:29.437852 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:29.437859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:29.438127 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 22:26:29.438337 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 22:26:29.438690 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":124,"threads_started":1}
I20260811 22:26:29.440824 4078 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42449
I20260811 22:26:29.441103 4078 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:26:29.441749 4078 heartbeater.cc:511] Master 127.24.71.254:42449 requested a full tablet report, sending...
I20260811 22:26:29.442911 3727 ts_manager.cc:194] Registered new tserver with Master: 2383151068aa41549dd7f04bc78f73ac (127.24.71.194:40223)
I20260811 22:26:29.443773 3727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:52851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:26:29.459743 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42449
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:26:29.579487 4083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:26:29.579788 4083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:26:29.579890 4083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:26:29.584043 4083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:26:29.584116 4083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:26:29.584183 4083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:26:29.589777 4083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:26:29.590932 4083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:26:29.592020 4083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:26:29.602695 4091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:29.602695 4092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:26:29.602728 4095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:26:29.602790 4083 server_base.cc:1034] running on GCE node
I20260811 22:26:29.603498 4083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:26:29.604342 4083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:26:29.605515 4083 hybrid_clock.cc:648] HybridClock initialized: now 1786487189605498 us; error 30 us; skew 500 ppm
Aug 11 22:26:29 dist-test-slave-mvvj krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:26:29.608623 4083 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:26:29.609570 4083 webserver.cc:460] Webserver started at http://127.24.71.195:42211/ using document root <none> and password file <none>
I20260811 22:26:29.610178 4083 fs_manager.cc:359] Metadata directory not provided
I20260811 22:26:29.610242 4083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:26:29.610471 4083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:26:29.612730 4083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "14fb2d8e450146fdb12cc9f76e5b0345"
format_stamp: "Formatted at 2026-08-11 22:26:29 on dist-test-slave-mvvj"
server_key: "c8d3c77bac46d5cd4910ad85aa0ef8e5"
server_key_iv: "2515914c15dbb3fd0511c0950aacee4e"
server_key_version: "encryptionkey@0"
I20260811 22:26:29.613319 4083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "14fb2d8e450146fdb12cc9f76e5b0345"
format_stamp: "Formatted at 2026-08-11 22:26:29 on dist-test-slave-mvvj"
server_key: "c8d3c77bac46d5cd4910ad85aa0ef8e5"
server_key_iv: "2515914c15dbb3fd0511c0950aacee4e"
server_key_version: "encryptionkey@0"
I20260811 22:26:29.617231 4083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:26:29.619784 4102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:29.621157 4083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:26:29.621258 4083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "14fb2d8e450146fdb12cc9f76e5b0345"
format_stamp: "Formatted at 2026-08-11 22:26:29 on dist-test-slave-mvvj"
server_key: "c8d3c77bac46d5cd4910ad85aa0ef8e5"
server_key_iv: "2515914c15dbb3fd0511c0950aacee4e"
server_key_version: "encryptionkey@0"
I20260811 22:26:29.621392 4083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:26:29.641101 4083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:26:29.644611 4083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:26:29.644824 4083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:26:29.645473 4083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:26:29.646335 4083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:26:29.646407 4083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:29.646478 4083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:26:29.646530 4083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:26:29.655814 4083 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:40193
I20260811 22:26:29.655874 4226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:40193 every 8 connection(s)
I20260811 22:26:29.656872 4083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:26:29.658443 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 4083
I20260811 22:26:29.658583 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:26:29.658898 24863 external_mini_cluster.cc:1428] Setting key e2f9ed51866cffe7633a87af8024d2cf
Aug 11 22:26:29 dist-test-slave-mvvj krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:26:29.671273 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.659993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:33729 (local address 127.24.71.254:42449)
0811 22:26:29.660137 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.660142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.660157 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:26:29.661738 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.661740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.661787 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.661855 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:29.663123 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.663658 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.664868 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.665096 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.665178 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.667692 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:29.667695 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:29.667697 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:29.667721 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:29.669208 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:29.669722 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:29.669725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:29.669726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:29.669816 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:29.670265 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:29.670267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:29.670268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:29.670457 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:29.670545 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.671036 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.671146 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:26:29.672261 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.659266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42449 (local address 127.24.71.195:33729)
0811 22:26:29.659911 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:29.659990 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:29.661537 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:29.661996 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:29.662003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:29.662387 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:26:29.662990 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.663005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.663773 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.663776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.664756 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.664767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.665234 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.665237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.665386 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.666026 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:26:29.666043 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:26:29.667530 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:26:29.669326 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:29.669331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:29.669344 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:29.669610 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:29.669979 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:26:29.669982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:26:29.669986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:26:29.670168 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:26:29.670552 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:26:29.670557 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:26:29.670801 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 22:26:29.670976 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 22:26:29.671248 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":156,"threads_started":1}
I20260811 22:26:29.673573 4227 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42449
I20260811 22:26:29.673823 4227 heartbeater.cc:464] Registering TS with master...
I20260811 22:26:29.674328 4227 heartbeater.cc:511] Master 127.24.71.254:42449 requested a full tablet report, sending...
I20260811 22:26:29.675423 3726 ts_manager.cc:194] Registered new tserver with Master: 14fb2d8e450146fdb12cc9f76e5b0345 (127.24.71.195:40193)
I20260811 22:26:29.676080 3726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:33729
I20260811 22:26:29.685658 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:26:29.695719 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.687801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41288 (local address 127.24.71.254:42449)
0811 22:26:29.687993 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.687998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.688127 (+ 129us) server_negotiation.cc:407] Connection header received
0811 22:26:29.688242 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.688244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.688290 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.688359 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:26:29.689089 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.689602 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.690501 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.690803 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.690887 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.692446 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:26:29.692449 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:26:29.692451 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:26:29.692477 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:26:29.693958 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:29.694426 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:29.694429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:29.694430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:29.694556 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:26:29.694809 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:26:29.694813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:26:29.694814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:26:29.695058 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:26:29.695165 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.695400 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.695507 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:26:29.699333 3726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:26:29.702711 3726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:26:29.719025 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.712619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:40223 (local address 127.0.0.1:53690)
0811 22:26:29.713427 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:29.713487 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:29.713636 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:29.714405 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:29.714410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:29.714427 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:29.714536 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.714540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.716064 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.716068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.717543 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.717553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.718424 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.718428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.718550 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.718589 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:26:29.718713 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 22:26:29.718851 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":115,"thread_start_us":79,"threads_started":1}
I20260811 22:26:29.719029 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.712790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.24.71.194:40223)
0811 22:26:29.713202 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.713216 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.713785 (+ 569us) server_negotiation.cc:407] Connection header received
0811 22:26:29.713868 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.713874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.714096 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.714250 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:29.714653 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.715904 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.717678 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.718277 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.718426 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.718596 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.718756 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.718858 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":174,"threads_started":1}
I20260811 22:26:29.719370 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.713373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:40193 (local address 127.0.0.1:50914)
0811 22:26:29.714187 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:29.714199 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:29.714269 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:29.715231 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:29.715234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:29.715252 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:29.715359 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.715369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.716648 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.716652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.717738 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.717747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.718828 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.718832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.718980 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.719063 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 22:26:29.719127 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 22:26:29.719207 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":43,"thread_start_us":103,"threads_started":1}
I20260811 22:26:29.719400 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.712790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40465 (local address 127.0.0.1:40926)
0811 22:26:29.714109 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:29.714125 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:29.714220 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:29.714655 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:29.714659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:29.714679 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:29.714777 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.714782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.716201 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.716205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.717503 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.717521 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.718834 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.718837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.719004 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.719060 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 22:26:29.719128 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 22:26:29.719207 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1246,"mutex_wait_us":104,"thread_start_us":101,"threads_started":1}
I20260811 22:26:29.719555 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.713816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.24.71.193:40465)
0811 22:26:29.714142 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.714151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.714179 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:26:29.714363 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.714368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.714523 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.714668 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:29.714898 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.716079 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.717654 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.718504 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.718625 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.718782 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.719294 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.719394 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":105,"threads_started":1}
I20260811 22:26:29.720405 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.713552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50914 (local address 127.24.71.195:40193)
0811 22:26:29.714836 (+ 1284us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.714849 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.714873 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:26:29.714954 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.714957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.715099 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.715253 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:29.715490 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.716500 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.717871 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.718691 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.719937 (+ 1246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.720108 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.720161 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.720256 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1177,"thread_start_us":113,"threads_started":1}
I20260811 22:26:29.722411 4007 tablet_service.cc:1467] Processing CreateTablet for tablet 36fefa465f2a4d4fbbc82a907e7b09d0 (DEFAULT_TABLE table=test-table [id=64889dad5ed0480a988105f9976db80e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:29.722800 4154 tablet_service.cc:1467] Processing CreateTablet for tablet 36fefa465f2a4d4fbbc82a907e7b09d0 (DEFAULT_TABLE table=test-table [id=64889dad5ed0480a988105f9976db80e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:29.722702 3860 tablet_service.cc:1467] Processing CreateTablet for tablet 36fefa465f2a4d4fbbc82a907e7b09d0 (DEFAULT_TABLE table=test-table [id=64889dad5ed0480a988105f9976db80e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:26:29.723404 4007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36fefa465f2a4d4fbbc82a907e7b09d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:29.723739 4154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36fefa465f2a4d4fbbc82a907e7b09d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:29.723737 3860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36fefa465f2a4d4fbbc82a907e7b09d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:26:29.731012 4246 tablet_bootstrap.cc:492] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: Bootstrap starting.
I20260811 22:26:29.732209 4244 tablet_bootstrap.cc:492] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: Bootstrap starting.
I20260811 22:26:29.734412 4246 tablet_bootstrap.cc:654] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:29.735034 4244 tablet_bootstrap.cc:654] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:29.735507 4246 log.cc:824] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:29.735914 4244 log.cc:824] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:29.736452 4245 tablet_bootstrap.cc:492] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e: Bootstrap starting.
I20260811 22:26:29.738065 4244 tablet_bootstrap.cc:492] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: No bootstrap required, opened a new log
I20260811 22:26:29.738081 4246 tablet_bootstrap.cc:492] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: No bootstrap required, opened a new log
I20260811 22:26:29.738288 4244 ts_tablet_manager.cc:1397] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:26:29.738288 4246 ts_tablet_manager.cc:1397] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:26:29.738654 4245 tablet_bootstrap.cc:654] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e: Neither blocks nor log segments found. Creating new log.
I20260811 22:26:29.739382 4245 log.cc:824] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:26:29.741348 4245 tablet_bootstrap.cc:492] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e: No bootstrap required, opened a new log
I20260811 22:26:29.741585 4245 ts_tablet_manager.cc:1397] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:26:29.742460 4244 raft_consensus.cc:348] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } }
I20260811 22:26:29.742460 4246 raft_consensus.cc:348] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } }
I20260811 22:26:29.742754 4244 raft_consensus.cc:374] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:29.742815 4244 raft_consensus.cc:729] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2383151068aa41549dd7f04bc78f73ac, State: Initialized, Role: FOLLOWER
I20260811 22:26:29.743093 4246 raft_consensus.cc:374] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:29.743170 4246 raft_consensus.cc:729] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14fb2d8e450146fdb12cc9f76e5b0345, State: Initialized, Role: FOLLOWER
I20260811 22:26:29.743350 4244 consensus_queue.cc:260] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } }
I20260811 22:26:29.743572 4246 consensus_queue.cc:260] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } }
I20260811 22:26:29.744460 4244 ts_tablet_manager.cc:1428] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:26:29.744498 4246 ts_tablet_manager.cc:1428] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:26:29.744881 4227 heartbeater.cc:503] Master 127.24.71.254:42449 was elected leader, sending a full tablet report...
I20260811 22:26:29.746003 4078 heartbeater.cc:503] Master 127.24.71.254:42449 was elected leader, sending a full tablet report...
I20260811 22:26:29.746029 4245 raft_consensus.cc:348] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } }
I20260811 22:26:29.746300 4245 raft_consensus.cc:374] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:26:29.746368 4245 raft_consensus.cc:729] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b8845c46e4046be9c70fe72f248b76e, State: Initialized, Role: FOLLOWER
I20260811 22:26:29.746799 4245 consensus_queue.cc:260] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } }
I20260811 22:26:29.747471 3932 heartbeater.cc:503] Master 127.24.71.254:42449 was elected leader, sending a full tablet report...
I20260811 22:26:29.747882 4245 ts_tablet_manager.cc:1428] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 22:26:29.863349 4251 raft_consensus.cc:482] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:26:29.863610 4251 raft_consensus.cc:504] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } }
I20260811 22:26:29.864950 4251 leader_election.cc:288] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2383151068aa41549dd7f04bc78f73ac (127.24.71.194:40223), 5b8845c46e4046be9c70fe72f248b76e (127.24.71.193:40465)
I20260811 22:26:29.869762 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.865479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:46867 (local address 127.24.71.193:40465)
0811 22:26:29.865618 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.865625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.865647 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:26:29.865878 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.865884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.865966 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.866067 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:29.866516 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.867405 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.868702 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.869150 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.869297 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.869360 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.869563 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.869646 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:26:29.869762 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.865322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40465 (local address 127.24.71.195:46867)
0811 22:26:29.865572 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:29.865589 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:29.865715 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:29.866107 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:29.866110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:29.866132 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:29.866295 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.866301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.867573 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.867576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.868552 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.868563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.869289 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.869293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.869424 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.869443 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:26:29.869533 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:26:29.869614 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"spinlock_wait_cycles":24704}
I20260811 22:26:29.869891 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.865490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:47417 (local address 127.24.71.194:40223)
0811 22:26:29.865652 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.865659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.866119 (+ 460us) server_negotiation.cc:407] Connection header received
0811 22:26:29.866334 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.866340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.866405 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.866504 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:26:29.866786 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.867582 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.868754 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.869209 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.869358 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.869420 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.869637 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.869740 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:26:29.869980 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.865322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:40223 (local address 127.24.71.195:47417)
0811 22:26:29.866005 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:29.866016 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:26:29.866196 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:26:29.866517 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:26:29.866520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:26:29.866538 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:26:29.866650 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.866657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.867721 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.867726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.868607 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:26:29.868618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.869353 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.869357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:26:29.869496 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.869517 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:26:29.869617 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:26:29.869740 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":165,"spinlock_wait_cycles":768,"thread_start_us":223,"threads_started":1}
I20260811 22:26:29.870728 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36fefa465f2a4d4fbbc82a907e7b09d0" candidate_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8845c46e4046be9c70fe72f248b76e" is_pre_election: true
I20260811 22:26:29.870725 4027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36fefa465f2a4d4fbbc82a907e7b09d0" candidate_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2383151068aa41549dd7f04bc78f73ac" is_pre_election: true
I20260811 22:26:29.871080 4027 raft_consensus.cc:2454] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14fb2d8e450146fdb12cc9f76e5b0345 in term 0.
I20260811 22:26:29.871096 3882 raft_consensus.cc:2454] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14fb2d8e450146fdb12cc9f76e5b0345 in term 0.
I20260811 22:26:29.871628 4106 leader_election.cc:302] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14fb2d8e450146fdb12cc9f76e5b0345, 2383151068aa41549dd7f04bc78f73ac, 5b8845c46e4046be9c70fe72f248b76e; no voters:
I20260811 22:26:29.871985 4251 raft_consensus.cc:2781] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:26:29.872097 4251 raft_consensus.cc:482] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:26:29.872143 4251 raft_consensus.cc:3037] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:29.873270 4251 raft_consensus.cc:504] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } }
I20260811 22:26:29.873679 4251 leader_election.cc:288] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [CANDIDATE]: Term 1 election: Requested vote from peers 2383151068aa41549dd7f04bc78f73ac (127.24.71.194:40223), 5b8845c46e4046be9c70fe72f248b76e (127.24.71.193:40465)
I20260811 22:26:29.874157 4027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36fefa465f2a4d4fbbc82a907e7b09d0" candidate_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2383151068aa41549dd7f04bc78f73ac"
I20260811 22:26:29.874158 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36fefa465f2a4d4fbbc82a907e7b09d0" candidate_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b8845c46e4046be9c70fe72f248b76e"
I20260811 22:26:29.874275 4027 raft_consensus.cc:3037] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:29.874310 3882 raft_consensus.cc:3037] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:26:29.875434 4027 raft_consensus.cc:2454] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14fb2d8e450146fdb12cc9f76e5b0345 in term 1.
I20260811 22:26:29.875600 3882 raft_consensus.cc:2454] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14fb2d8e450146fdb12cc9f76e5b0345 in term 1.
I20260811 22:26:29.875805 4106 leader_election.cc:302] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14fb2d8e450146fdb12cc9f76e5b0345, 2383151068aa41549dd7f04bc78f73ac; no voters:
I20260811 22:26:29.876060 4251 raft_consensus.cc:2781] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:26:29.876348 4251 raft_consensus.cc:686] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 1 LEADER]: Becoming Leader. State: Replica: 14fb2d8e450146fdb12cc9f76e5b0345, State: Running, Role: LEADER
I20260811 22:26:29.876754 4251 consensus_queue.cc:237] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } }
I20260811 22:26:29.880424 3727 catalog_manager.cc:5495] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14fb2d8e450146fdb12cc9f76e5b0345 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14fb2d8e450146fdb12cc9f76e5b0345" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 40193 } health_report { overall_health: HEALTHY } } }
W20260811 22:26:29.908025 4228 tablet.cc:2369] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:26:29.913465 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.909257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50924 (local address 127.24.71.195:40193)
0811 22:26:29.909449 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:26:29.909453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:26:29.909466 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:26:29.909534 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:26:29.909536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:26:29.909592 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:26:29.909680 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:26:29.909975 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.910618 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.911783 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:26:29.912108 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:26:29.912203 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:26:29.912522 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:26:29.912954 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:26:29.913056 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:26:29.913152 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 22:26:29.913210 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 22:26:29.919341 3882 raft_consensus.cc:1261] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [term 1 FOLLOWER]: Refusing update from remote peer 14fb2d8e450146fdb12cc9f76e5b0345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:29.919341 4027 raft_consensus.cc:1261] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [term 1 FOLLOWER]: Refusing update from remote peer 14fb2d8e450146fdb12cc9f76e5b0345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:26:29.920296 4251 consensus_queue.cc:1035] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2383151068aa41549dd7f04bc78f73ac" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:26:29.920555 4254 consensus_queue.cc:1035] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b8845c46e4046be9c70fe72f248b76e" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:26:29.923604 4079 tablet.cc:2369] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:26:29.932462 4259 mvcc.cc:205] Tried to move back new op lower bound from 7317451529902276608 to 7317451529738309632. Current Snapshot: MvccSnapshot[applied={T|T < 7317451529902276608}]
I20260811 22:26:29.939333 3727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:29.939554 3727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:26:29.942771 3727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13328d08e633435db5dd90667e57d2ac: Sending DeleteTablet for 3 replicas of tablet 36fefa465f2a4d4fbbc82a907e7b09d0
I20260811 22:26:29.943781 4007 tablet_service.cc:1514] Processing DeleteTablet for tablet 36fefa465f2a4d4fbbc82a907e7b09d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:29 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:53690
I20260811 22:26:29.943922 3860 tablet_service.cc:1514] Processing DeleteTablet for tablet 36fefa465f2a4d4fbbc82a907e7b09d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:29 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:40926
I20260811 22:26:29.944065 4154 tablet_service.cc:1514] Processing DeleteTablet for tablet 36fefa465f2a4d4fbbc82a907e7b09d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:26:29 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:50914
I20260811 22:26:29.944650 4268 tablet_replica.cc:330] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: stopping tablet replica
I20260811 22:26:29.944649 4269 tablet_replica.cc:330] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e: stopping tablet replica
I20260811 22:26:29.945061 4269 raft_consensus.cc:2229] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:29.945063 4268 raft_consensus.cc:2229] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:26:29.945263 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 3794
I20260811 22:26:29.945441 4268 raft_consensus.cc:2258] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:29.945581 4269 raft_consensus.cc:2258] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 5b8845c46e4046be9c70fe72f248b76e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:29.946060 4270 tablet_replica.cc:330] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: stopping tablet replica
I20260811 22:26:29.946393 4270 raft_consensus.cc:2229] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:26:29.946794 4270 raft_consensus.cc:2258] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:26:29.948094 4268 ts_tablet_manager.cc:1905] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:29.950165 4270 ts_tablet_manager.cc:1905] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:26:29.952140 4268 ts_tablet_manager.cc:1918] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:26:29.952345 4268 log.cc:1196] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/36fefa465f2a4d4fbbc82a907e7b09d0
I20260811 22:26:29.952786 4268 ts_tablet_manager.cc:1939] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 2383151068aa41549dd7f04bc78f73ac: Deleting consensus metadata
I20260811 22:26:29.953166 4270 ts_tablet_manager.cc:1918] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:26:29.953271 4270 log.cc:1196] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/36fefa465f2a4d4fbbc82a907e7b09d0
I20260811 22:26:29.953629 4270 ts_tablet_manager.cc:1939] T 36fefa465f2a4d4fbbc82a907e7b09d0 P 14fb2d8e450146fdb12cc9f76e5b0345: Deleting consensus metadata
I20260811 22:26:29.954679 3714 catalog_manager.cc:4841] TS 2383151068aa41549dd7f04bc78f73ac (127.24.71.194:40223): tablet 36fefa465f2a4d4fbbc82a907e7b09d0 (table test-table [id=64889dad5ed0480a988105f9976db80e]) successfully deleted
I20260811 22:26:29.954782 3711 catalog_manager.cc:4841] TS 14fb2d8e450146fdb12cc9f76e5b0345 (127.24.71.195:40193): tablet 36fefa465f2a4d4fbbc82a907e7b09d0 (table test-table [id=64889dad5ed0480a988105f9976db80e]) successfully deleted
W20260811 22:26:29.958190 3711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:40465 (error 108)
I20260811 22:26:29.958406 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 3937
I20260811 22:26:29.964433 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:26:29.961373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40465 (local address 127.0.0.1:40932)
0811 22:26:29.964189 (+ 2816us) negotiation.cc:107] Waiting for socket to connect
0811 22:26:29.964299 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:40465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2677}
W20260811 22:26:29.964722 3711 catalog_manager.cc:4584] TS 5b8845c46e4046be9c70fe72f248b76e (127.24.71.193:40465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 36fefa465f2a4d4fbbc82a907e7b09d0: Network error: Client connection negotiation failed: client connection to 127.24.71.193:40465: connect: Connection refused (error 111)
I20260811 22:26:29.971041 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 4083
I20260811 22:26:29.978993 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 3693
2026-08-11T22:26:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2591 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:26:30 dist-test-slave-mvvj krb5kdc[4274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:26:30 dist-test-slave-mvvj krb5kdc[4274](info): set up 2 sockets
Aug 11 22:26:30 dist-test-slave-mvvj krb5kdc[4274](info): commencing operation
krb5kdc: starting...
W20260811 22:26:31.171316 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:26:31 dist-test-slave-mvvj krb5kdc[4274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:26:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:26:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.71.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.71.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/rangeradmin_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/rangeradmin_127.24.71.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.71.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.71.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/rangerlookup_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/rangerlookup_127.24.71.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.71.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.71.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/HTTP_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/HTTP_127.24.71.212@KRBTEST.COM.keytab.
I20260811 22:26:31.367264 24863 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskapOnzC/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/postgres -l logfile start
2026-08-11 22:26:34.183 UTC [4308] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:26:34.183 UTC [4308] LOG: listening on IPv4 address "127.24.71.212", port 58983
2026-08-11 22:26:34.187 UTC [4308] LOG: listening on Unix socket "/tmp/.s.PGSQL.58983"
2026-08-11 22:26:34.212 UTC [4312] LOG: database system was shut down at 2026-08-11 22:26:32 UTC
2026-08-11 22:26:34.217 UTC [4308] LOG: database system is ready to accept connections
I20260811 22:26:35.227217 24863 mini_postgres.cc:96] Postgres bound to 58983
2026-08-11 22:26:35.235 UTC [4322] FATAL: database "slave" does not exist
127.24.71.212:58983 - accepting connections
I20260811 22:26:35.236011 24863 mini_ranger.cc:161] Starting Ranger...
I20260811 22:26:35.253590 24863 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:26:35.805055 24863 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:26:35.805174 24863 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-admin
2026-08-11 22:26:36,121 [I] DB FLAVOR :POSTGRES
2026-08-11 22:26:36,122 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:26:36,122 [I] Checking connection..
2026-08-11 22:26:36,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:26:36,382 [I] Checking connection passed.
2026-08-11 22:26:36,382 [I] --------- Verifying version history table ---------
2026-08-11 22:26:36,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:26:36,648 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:26:36,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:26:36,900 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:26:36,900 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:26:36,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:26:37.163 UTC [4394] WARNING: there is no transaction in progress
2026-08-11 22:26:37,171 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:26:37,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:26:37,427 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:26:37,428 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:26:37,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:26:37,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:26:37,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mvvj.c.gcp-upstream.internal','N') ;"
2026-08-11 22:26:38,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:26:38,476 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:26:38,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:26:38,731 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:26:38,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:26:38,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:26:39,246 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:26:39,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:26:40.285 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.303 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.322 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.338 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.355 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.375 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.435 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.442 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.458 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.469 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.479 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.494 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.505 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.512 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.519 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:40.527 UTC [4512] WARNING: there is no transaction in progress
2026-08-11 22:26:41,277 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:26:41,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:26:41,540 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:26:41,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:26:41,784 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:26:41,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:26:42,045 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:26:42,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:26:42,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:26:42,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mvvj.c.gcp-upstream.internal';"
2026-08-11 22:26:42,822 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:26:42,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:26:43,092 [I] DB_PATCHES have already been applied
I20260811 22:26:43.098894 24863 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-admin:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskapOnzC/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:26:43.098966 24863 mini_ranger.cc:193] Using host: 127.24.71.212
I20260811 22:26:43.103729 24863 mini_ranger.cc:227] Ranger admin URL: http://127.24.71.212:33409
Aug 11, 2026 10:26:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:26:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:26:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:26:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:26:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:26:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:26:44 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:26:44.300849 24863 mini_ranger.cc:160] Time spent starting Ranger: real 9.065s user 0.006s sys 0.008s
Aug 11, 2026 10:27:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:27:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:27:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:27:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:27:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:27:07.804225 24863 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.71.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.71.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/rangerkms_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/rangerkms_127.24.71.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.71.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.71.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.71.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/HTTP_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.71.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/HTTP_127.24.71.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:27:07 dist-test-slave-mvvj krb5kdc[4274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487227, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:27:07.916536 24863 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:27:07.929893 24863 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:27:09.000339 24863 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:27:09.000456 24863 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-kms
I20260811 22:27:09.000586 24863 mini_ranger_kms.cc:227] Using postgres at 127.24.71.212:58983
2026-08-11 22:27:09,404 [I] DB FLAVOR :POSTGRES
2026-08-11 22:27:09,404 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:27:09,404 [I] Checking connection
2026-08-11 22:27:09,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:27:09,711 [I] connection success
2026-08-11 22:27:09,711 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:27:09,712 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:27:09,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:27:10,019 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:27:10,019 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:27:10,019 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:27:10,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:58983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:27:10,338 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:27:10.862619 24863 mini_ranger_kms.cc:310] Created kms service
I20260811 22:27:11.095280 24863 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:27:11.139595 24863 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:27:11 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487227, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:27:11.591817 24863 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:27:11.592219 24863 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-kms:/tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:27:11.592260 24863 mini_ranger_kms.cc:240] Using host: 127.24.71.212
I20260811 22:27:11.598274 24863 mini_ranger_kms.cc:276] Ranger KMS PID: 4710
I20260811 22:27:11.598484 24863 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.71.212:50147
Aug 11, 2026 10:27:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:27:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:27:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:27:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:27:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:27:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 22:27:14.089788 24863 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.173s user 0.000s sys 0.380s
I20260811 22:27:14.089938 24863 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:27:14.090049 24863 mini_ranger_kms.cc:401] 127.24.71.212:50147/kms/v1/keys
Aug 11 22:27:14 dist-test-slave-mvvj krb5kdc[4274](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487234, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.71.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 22:27:14 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487234, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.71.212@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
Aug 11 22:27:14 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487234, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.71.212@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
Aug 11 22:27:17 dist-test-slave-mvvj krb5kdc[4274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:27:17.331496 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40281
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:39931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:40281
--ranger_config_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:50147/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:27:17.467628 4766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:27:17.468021 4766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:27:17.468101 4766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:27:17.472499 4766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:27:17.472596 4766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:27:17.472635 4766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:27:17.472682 4766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:27:17.472729 4766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:27:17.478727 4766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:39931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.71.212:50147/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:40281
--ranger_config_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:27:17.480062 4766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:27:17.481251 4766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:27:17.490916 4773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:27:17.491089 4772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:27:17.490916 4775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:27:17.491510 4766 server_base.cc:1034] running on GCE node
I20260811 22:27:17.492352 4766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:27:17.494032 4766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:27:17.495249 4766 hybrid_clock.cc:648] HybridClock initialized: now 1786487237495216 us; error 44 us; skew 500 ppm
Aug 11 22:27:17 dist-test-slave-mvvj krb5kdc[4274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:27:17.502177 4766 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:27:17.503216 4766 webserver.cc:460] Webserver started at http://127.24.71.254:39631/ using document root <none> and password file <none>
I20260811 22:27:17.503818 4766 fs_manager.cc:359] Metadata directory not provided
I20260811 22:27:17.503966 4766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:27:17.504207 4766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:27:17 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:27:17.598685 4766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "d890dd6d9fc34250859534ad7e08fa83"
format_stamp: "Formatted at 2026-08-11 22:27:17 on dist-test-slave-mvvj"
server_key: "2c36a50a58c12178f46343c93fe71e32"
server_key_iv: "70f3eaef4b2d19391611d9dd6596d242"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:17.599444 4766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "d890dd6d9fc34250859534ad7e08fa83"
format_stamp: "Formatted at 2026-08-11 22:27:17 on dist-test-slave-mvvj"
server_key: "2c36a50a58c12178f46343c93fe71e32"
server_key_iv: "70f3eaef4b2d19391611d9dd6596d242"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:17.603688 4766 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:27:17.622787 4784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:17.624416 4766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:27:17.624565 4766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "d890dd6d9fc34250859534ad7e08fa83"
format_stamp: "Formatted at 2026-08-11 22:27:17 on dist-test-slave-mvvj"
server_key: "2c36a50a58c12178f46343c93fe71e32"
server_key_iv: "70f3eaef4b2d19391611d9dd6596d242"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:17.624714 4766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:27:17.655428 4766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:27:17.658746 4766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:27:17.658981 4766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:27:17.667676 4766 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:40281
I20260811 22:27:17.667719 4843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:40281 every 8 connection(s)
I20260811 22:27:17.668817 4766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:27:17.671953 4844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:27:17.674321 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 4766
I20260811 22:27:17.674451 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:27:17.674793 24863 external_mini_cluster.cc:1428] Setting key 061c8f2072eb0b52de4969e315cd3418
I20260811 22:27:17.679550 4844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83: Bootstrap starting.
I20260811 22:27:17.682500 4844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83: Neither blocks nor log segments found. Creating new log.
I20260811 22:27:17.683543 4844 log.cc:824] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83: Log is configured to *not* fsync() on all Append() calls
I20260811 22:27:17.685913 4844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83: No bootstrap required, opened a new log
Aug 11 22:27:17 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:27:17.691187 4844 raft_consensus.cc:348] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890dd6d9fc34250859534ad7e08fa83" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40281 } }
I20260811 22:27:17.691529 4844 raft_consensus.cc:374] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:27:17.691596 4844 raft_consensus.cc:729] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d890dd6d9fc34250859534ad7e08fa83, State: Initialized, Role: FOLLOWER
I20260811 22:27:17.692142 4844 consensus_queue.cc:260] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890dd6d9fc34250859534ad7e08fa83" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40281 } }
I20260811 22:27:17.692327 4844 raft_consensus.cc:388] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:27:17.692397 4844 raft_consensus.cc:482] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:27:17.692513 4844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:27:17.693636 4844 raft_consensus.cc:504] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890dd6d9fc34250859534ad7e08fa83" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40281 } }
I20260811 22:27:17.694021 4844 leader_election.cc:302] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d890dd6d9fc34250859534ad7e08fa83; no voters:
I20260811 22:27:17.694365 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:17.676533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43558 (local address 127.24.71.254:40281)
0811 22:27:17.677101 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:17.677124 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:17.677174 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:27:17.678158 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:17.678185 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:17.678675 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:17.679097 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:27:17.680133 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:17.681629 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:17.682784 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:17.683529 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:17.683786 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:17.687895 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:27:17.687908 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:27:17.687940 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:27:17.687977 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:27:17.690173 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:17.690841 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:17.690848 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:17.690855 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:17.691011 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:17.691367 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:17.691372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:17.691373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:17.691976 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:27:17.692188 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:17.692896 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 22:27:17.693226 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":167,"threads_started":1}
I20260811 22:27:17.694451 4844 leader_election.cc:288] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:27:17.694890 4849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:27:17.695145 4849 raft_consensus.cc:686] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [term 1 LEADER]: Becoming Leader. State: Replica: d890dd6d9fc34250859534ad7e08fa83, State: Running, Role: LEADER
I20260811 22:27:17.695546 4849 consensus_queue.cc:237] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890dd6d9fc34250859534ad7e08fa83" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40281 } }
I20260811 22:27:17.696954 4849 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d890dd6d9fc34250859534ad7e08fa83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890dd6d9fc34250859534ad7e08fa83" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40281 } } }
I20260811 22:27:17.697139 4844 sys_catalog.cc:564] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:27:17.697161 4849 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [sys.catalog]: This master's current role is: LEADER
I20260811 22:27:17.697468 4849 sys_catalog.cc:455] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d890dd6d9fc34250859534ad7e08fa83. Latest consensus state: current_term: 1 leader_uuid: "d890dd6d9fc34250859534ad7e08fa83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d890dd6d9fc34250859534ad7e08fa83" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40281 } } }
I20260811 22:27:17.697647 4849 sys_catalog.cc:458] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83 [sys.catalog]: This master's current role is: LEADER
I20260811 22:27:17.697683 4853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:27:17.701246 4853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:27:17.701813 4844 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:27:17.714823 4853 catalog_manager.cc:1347] Generated new cluster ID: 851d307dda63458180d0e5d941c23e67
I20260811 22:27:17.714926 4853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:27:17.737958 4853 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:27:17.740128 4853 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:27:17.751446 4853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83: Generated new TSK 0
I20260811 22:27:17.752511 4853 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:27:18.487145 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:50147/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40281
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:39931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:27:18.640861 4893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:27:18.641201 4893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:27:18.641322 4893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:27:18.646373 4893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:27:18.646479 4893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:27:18.646570 4893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:27:18.655041 4893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:39931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.71.212:50147/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:27:18.656635 4893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:27:18.658107 4893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:27:18.669782 4901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:27:18.670215 4902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:27:18.670462 4904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:27:18.671777 4893 server_base.cc:1034] running on GCE node
I20260811 22:27:18.672358 4893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:27:18.673295 4893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:27:18.676592 4893 hybrid_clock.cc:648] HybridClock initialized: now 1786487238676495 us; error 100 us; skew 500 ppm
Aug 11 22:27:18 dist-test-slave-mvvj krb5kdc[4274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487238, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:27:18.681137 4893 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:27:18.682528 4893 webserver.cc:460] Webserver started at http://127.24.71.193:33671/ using document root <none> and password file <none>
I20260811 22:27:18.683264 4893 fs_manager.cc:359] Metadata directory not provided
I20260811 22:27:18.683355 4893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:27:18.683801 4893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:27:18 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487238, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:27:18.714537 4893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "a1bb4d55cba648c890ba2c1de267f54a"
format_stamp: "Formatted at 2026-08-11 22:27:18 on dist-test-slave-mvvj"
server_key: "96f7098801ea27f29e060de6a33a0aa7"
server_key_iv: "69e2a18219f0d1bf56fda54b9b1d3a83"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:18.715605 4893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "a1bb4d55cba648c890ba2c1de267f54a"
format_stamp: "Formatted at 2026-08-11 22:27:18 on dist-test-slave-mvvj"
server_key: "96f7098801ea27f29e060de6a33a0aa7"
server_key_iv: "69e2a18219f0d1bf56fda54b9b1d3a83"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:18.721835 4893 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.003s
I20260811 22:27:18.750310 4914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:18.752255 4893 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.001s
I20260811 22:27:18.752502 4893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "a1bb4d55cba648c890ba2c1de267f54a"
format_stamp: "Formatted at 2026-08-11 22:27:18 on dist-test-slave-mvvj"
server_key: "96f7098801ea27f29e060de6a33a0aa7"
server_key_iv: "69e2a18219f0d1bf56fda54b9b1d3a83"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:18.752770 4893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:27:18.773989 4893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:27:18.778066 4893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:27:18.778379 4893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:27:18.779250 4893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:27:18.780444 4893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:27:18.780565 4893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:18.780700 4893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:27:18.780782 4893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:18.796497 4893 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:46391
I20260811 22:27:18.796804 5038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:46391 every 8 connection(s)
I20260811 22:27:18.797833 4893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:27:18.798367 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 4893
I20260811 22:27:18.798513 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:27:18.798988 24863 external_mini_cluster.cc:1428] Setting key bcdd23a22bc00dd8b42c27cc8910208d
Aug 11 22:27:18 dist-test-slave-mvvj krb5kdc[4274](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487238, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 22:27:18 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487238, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:27:18.822710 5042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:18.805344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40281 (local address 127.24.71.193:39197)
0811 22:27:18.806010 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:18.806078 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:18.807323 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:18.808638 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:18.808648 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:18.809069 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:27:18.809765 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:18.809787 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:18.811022 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:18.811026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:18.812565 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:18.812578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:18.813428 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:18.813434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:18.813675 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:18.814425 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:27:18.814446 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:27:18.816354 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:27:18.818774 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:27:18.818781 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:27:18.818801 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:27:18.819179 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:27:18.819512 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:27:18.819515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:27:18.819518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:27:18.819773 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:27:18.820475 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:27:18.820482 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:27:18.820907 (+ 425us) client_negotiation.cc:769] Sending connection context
0811 22:27:18.821144 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 22:27:18.821541 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":171,"threads_started":1}
I20260811 22:27:18.823545 5044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:18.805637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:39197 (local address 127.24.71.254:40281)
0811 22:27:18.808056 (+ 2419us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:18.808061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:18.808079 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:27:18.808128 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:18.808132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:18.808185 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:18.808289 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:27:18.809966 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:18.810861 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:18.812721 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:18.813100 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:18.813208 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:18.816578 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:27:18.816582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:27:18.816585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:27:18.816615 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:27:18.818611 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:18.819307 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:18.819310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:18.819312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:18.819410 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:18.820059 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:18.820063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:18.820065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:18.820313 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:27:18.820413 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:18.823217 (+ 2804us) server_negotiation.cc:299] Negotiation successful
0811 22:27:18.823366 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2328,"thread_start_us":107,"threads_started":1}
I20260811 22:27:18.825687 5039 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40281
I20260811 22:27:18.825982 5039 heartbeater.cc:464] Registering TS with master...
I20260811 22:27:18.826606 5039 heartbeater.cc:511] Master 127.24.71.254:40281 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:27:18.828576 4801 ts_manager.cc:194] Registered new tserver with Master: a1bb4d55cba648c890ba2c1de267f54a (127.24.71.193:46391)
I20260811 22:27:18.830240 4801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:39197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:27:18.856045 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:50147/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40281
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:39931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:27:19.055706 5046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:27:19.056231 5046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:27:19.056393 5046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:27:19.063488 5046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:27:19.063627 5046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:27:19.063746 5046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:27:19.073164 5046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:39931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.71.212:50147/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:27:19.075054 5046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:27:19.076768 5046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:27:19.092010 5053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:27:19.092777 5052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:27:19.093137 5046 server_base.cc:1034] running on GCE node
W20260811 22:27:19.093835 5056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:27:19.094420 5046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:27:19.095304 5046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:27:19.096482 5046 hybrid_clock.cc:648] HybridClock initialized: now 1786487239096440 us; error 60 us; skew 500 ppm
Aug 11 22:27:19 dist-test-slave-mvvj krb5kdc[4274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487239, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:27:19.101105 5046 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:27:19.102378 5046 webserver.cc:460] Webserver started at http://127.24.71.194:39265/ using document root <none> and password file <none>
I20260811 22:27:19.103242 5046 fs_manager.cc:359] Metadata directory not provided
I20260811 22:27:19.103422 5046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:27:19.103752 5046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:27:19 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487239, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:27:19.122243 5046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "8074fd68d1e74c90ae366168740fd3ee"
format_stamp: "Formatted at 2026-08-11 22:27:19 on dist-test-slave-mvvj"
server_key: "66aa8fed6832ea3e3585de198ccccebe"
server_key_iv: "c39dd03e4428571cdc357b7e9a122744"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:19.122989 5046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "8074fd68d1e74c90ae366168740fd3ee"
format_stamp: "Formatted at 2026-08-11 22:27:19 on dist-test-slave-mvvj"
server_key: "66aa8fed6832ea3e3585de198ccccebe"
server_key_iv: "c39dd03e4428571cdc357b7e9a122744"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:19.128098 5046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:27:19.150513 5063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:19.152158 5046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:27:19.152333 5046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "8074fd68d1e74c90ae366168740fd3ee"
format_stamp: "Formatted at 2026-08-11 22:27:19 on dist-test-slave-mvvj"
server_key: "66aa8fed6832ea3e3585de198ccccebe"
server_key_iv: "c39dd03e4428571cdc357b7e9a122744"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:19.152488 5046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:27:19.177139 5046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:27:19.180685 5046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:27:19.180927 5046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:27:19.181648 5046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:27:19.182649 5046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:27:19.182721 5046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:19.182837 5046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:27:19.182883 5046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:19.200255 5046 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:43955
I20260811 22:27:19.200328 5185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:43955 every 8 connection(s)
I20260811 22:27:19.212976 5046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:27:19.219887 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 5046
I20260811 22:27:19.220050 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:27:19.220412 24863 external_mini_cluster.cc:1428] Setting key 4c80a5c74218c0141faff433a6e6e494
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
Aug 11 22:27:19 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487239, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:27:19.268869 5044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.228033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:60947 (local address 127.24.71.254:40281)
0811 22:27:19.228173 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:19.228178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:19.232125 (+ 3947us) server_negotiation.cc:407] Connection header received
0811 22:27:19.233482 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:19.233486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:19.233562 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:19.233649 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:27:19.244637 (+ 10988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.245490 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.247386 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.247792 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.247900 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.263631 (+ 15731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:27:19.263636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:27:19.263639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:27:19.263679 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:27:19.265682 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:19.266782 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:19.266787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:19.266791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:19.266905 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:19.267363 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:19.267366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:19.267368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:19.267643 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:27:19.267732 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:19.268513 (+ 781us) server_negotiation.cc:299] Negotiation successful
0811 22:27:19.268666 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:27:19.273241 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.228375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40281 (local address 127.24.71.194:60947)
0811 22:27:19.228934 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:19.231983 (+ 3049us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:19.233202 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:19.242944 (+ 9742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:19.242960 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:19.243448 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:27:19.244349 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.244376 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.245701 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.245705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.247211 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.247223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.248153 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.248161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.248402 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.249578 (+ 1176us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:27:19.249602 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:27:19.263297 (+ 13695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:27:19.266110 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:27:19.266118 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:27:19.266141 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:27:19.266620 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:27:19.267024 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:27:19.267027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:27:19.267029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:27:19.267240 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:27:19.267805 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:27:19.267813 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:27:19.268196 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 22:27:19.269371 (+ 1175us) client_negotiation.cc:240] Negotiation successful
0811 22:27:19.269775 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":186,"threads_started":1}
I20260811 22:27:19.276237 5186 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40281
I20260811 22:27:19.276520 5186 heartbeater.cc:464] Registering TS with master...
I20260811 22:27:19.277132 5186 heartbeater.cc:511] Master 127.24.71.254:40281 requested a full tablet report, sending...
I20260811 22:27:19.278422 4801 ts_manager.cc:194] Registered new tserver with Master: 8074fd68d1e74c90ae366168740fd3ee (127.24.71.194:43955)
I20260811 22:27:19.279201 4801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:60947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:27:19.315516 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:50147/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40281
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:39931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:27:19 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487238, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
W20260811 22:27:19.587694 5194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:27:19.588146 5194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:27:19.588222 5194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:27:19.594724 5194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:27:19.594825 5194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:27:19.594928 5194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:27:19.603538 5194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:39931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.71.212:50147/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:27:19.605378 5194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:27:19.606967 5194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:27:19.621953 5203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:27:19.622258 5194 server_base.cc:1034] running on GCE node
W20260811 22:27:19.622787 5201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:27:19.623070 5200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:27:19.623668 5194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:27:19.624583 5194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:27:19.625821 5194 hybrid_clock.cc:648] HybridClock initialized: now 1786487239625748 us; error 90 us; skew 500 ppm
Aug 11 22:27:19 dist-test-slave-mvvj krb5kdc[4274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487239, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:27:19.630409 5194 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:27:19.631698 5194 webserver.cc:460] Webserver started at http://127.24.71.195:36057/ using document root <none> and password file <none>
I20260811 22:27:19.632534 5194 fs_manager.cc:359] Metadata directory not provided
I20260811 22:27:19.632701 5194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:27:19.633035 5194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:27:19 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487239, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:27:19.658164 5194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "ba6946371edb40dda12cd6dd9aaa5992"
format_stamp: "Formatted at 2026-08-11 22:27:19 on dist-test-slave-mvvj"
server_key: "3bb7b4ae448b9540d08596175cd1cef0"
server_key_iv: "14038ecbaad2c4099cd9538a5033ad33"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:19.659258 5194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "ba6946371edb40dda12cd6dd9aaa5992"
format_stamp: "Formatted at 2026-08-11 22:27:19 on dist-test-slave-mvvj"
server_key: "3bb7b4ae448b9540d08596175cd1cef0"
server_key_iv: "14038ecbaad2c4099cd9538a5033ad33"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:19.666383 5194 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260811 22:27:19.702455 5212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:19.704213 5194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:27:19.704357 5194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "ba6946371edb40dda12cd6dd9aaa5992"
format_stamp: "Formatted at 2026-08-11 22:27:19 on dist-test-slave-mvvj"
server_key: "3bb7b4ae448b9540d08596175cd1cef0"
server_key_iv: "14038ecbaad2c4099cd9538a5033ad33"
server_key_version: "kuduclusterkey@0"
I20260811 22:27:19.704493 5194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:27:19.724505 5194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:27:19.728462 5194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:27:19.728791 5194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:27:19.729511 5194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:27:19.730660 5194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:27:19.730762 5194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:19.730880 5194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:27:19.730947 5194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:27:19.746893 5194 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:32797
I20260811 22:27:19.748186 5194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:27:19.750312 5337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:32797 every 8 connection(s)
I20260811 22:27:19.757380 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 5194
I20260811 22:27:19.757540 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:27:19.757884 24863 external_mini_cluster.cc:1428] Setting key 119d9e846ea1bf6afaafbc3d76fbe4da
Aug 11 22:27:19 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487239, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:27:19.777367 5044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.754331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:50271 (local address 127.24.71.254:40281)
0811 22:27:19.754461 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:19.754466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:19.755343 (+ 877us) server_negotiation.cc:407] Connection header received
0811 22:27:19.756598 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:19.756602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:19.756659 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:19.756710 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:27:19.758424 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.759255 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.761116 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.761416 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.761502 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.765077 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:27:19.765082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:27:19.765085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:27:19.765114 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:27:19.766906 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:19.767690 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:19.767694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:19.767695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:19.767786 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:19.772544 (+ 4758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:19.772547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:19.772549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:19.772797 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:27:19.772914 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:19.777023 (+ 4109us) server_negotiation.cc:299] Negotiation successful
0811 22:27:19.777200 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:27:19.778491 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.753100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40281 (local address 127.24.71.195:50271)
0811 22:27:19.755172 (+ 2072us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:19.755224 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:19.756334 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:19.757140 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:19.757149 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:19.757500 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:27:19.758211 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.758233 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.759424 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.759427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.760956 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.760969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.761733 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.761738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.762004 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.762758 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:27:19.762780 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:27:19.764862 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:27:19.767137 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:27:19.767144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:27:19.767162 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:27:19.767548 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:27:19.772016 (+ 4468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:27:19.772020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:27:19.772022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:27:19.772267 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:27:19.776244 (+ 3977us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:27:19.776254 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:27:19.776665 (+ 411us) client_negotiation.cc:769] Sending connection context
0811 22:27:19.776923 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 22:27:19.777328 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1562,"thread_start_us":204,"threads_started":1}
I20260811 22:27:19.779868 5338 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40281
I20260811 22:27:19.780121 5338 heartbeater.cc:464] Registering TS with master...
I20260811 22:27:19.780743 5338 heartbeater.cc:511] Master 127.24.71.254:40281 requested a full tablet report, sending...
I20260811 22:27:19.781903 4801 ts_manager.cc:194] Registered new tserver with Master: ba6946371edb40dda12cd6dd9aaa5992 (127.24.71.195:32797)
I20260811 22:27:19.782666 4801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:50271
I20260811 22:27:19.784466 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:27:19.804446 5044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.789026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.24.71.254:40281)
0811 22:27:19.789144 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:19.789149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:19.789823 (+ 674us) server_negotiation.cc:407] Connection header received
0811 22:27:19.790039 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:19.790042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:19.790091 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:19.790143 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:27:19.791725 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.792535 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.793780 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.794135 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.794208 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.796062 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:27:19.796066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:27:19.796069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:27:19.796097 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:27:19.801577 (+ 5480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:19.802811 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:19.802816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:19.802818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:19.802922 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:27:19.803298 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:27:19.803302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:27:19.803303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:27:19.803580 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:27:19.803723 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:19.804131 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 22:27:19.804276 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 22:27:19 dist-test-slave-mvvj krb5kdc[4274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487238, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:27:19.810541 4801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:27:19.815083 4801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:27:19.833257 5039 heartbeater.cc:503] Master 127.24.71.254:40281 was elected leader, sending a full tablet report...
I20260811 22:27:19.846475 5352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.836442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:46391 (local address 127.0.0.1:50868)
0811 22:27:19.840791 (+ 4349us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:19.840807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:19.840900 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:19.841593 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:19.841597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:19.841616 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:27:19.841706 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.841712 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.843365 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.843368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.844596 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.844607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.845925 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.845929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.846044 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.846089 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 22:27:19.846188 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 22:27:19.846317 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4286,"thread_start_us":87,"threads_started":1}
I20260811 22:27:19.847226 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.835935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:43955 (local address 127.0.0.1:38706)
0811 22:27:19.838657 (+ 2722us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:19.838673 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:19.838776 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:19.839404 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:19.839411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:19.839439 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:27:19.839658 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.839669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.843044 (+ 3375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.843047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.844172 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.844181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.846455 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.846459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.846861 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.846888 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:27:19.847001 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:27:19.847073 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2653,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260811 22:27:19.847800 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.836734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38706 (local address 127.24.71.194:43955)
0811 22:27:19.837069 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:19.837075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:19.838895 (+ 1820us) server_negotiation.cc:407] Connection header received
0811 22:27:19.838987 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:19.838993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:19.839207 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:19.839389 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:27:19.839825 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.842892 (+ 3067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.844452 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.845240 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.845354 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.845518 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:19.847473 (+ 1955us) server_negotiation.cc:299] Negotiation successful
0811 22:27:19.847611 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":146,"threads_started":1}
I20260811 22:27:19.848400 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.835301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:32797 (local address 127.0.0.1:54490)
0811 22:27:19.836328 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:19.836386 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:19.836535 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:19.840562 (+ 4027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:19.840566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:19.840607 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:27:19.840733 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.840741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.842527 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.842531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.844011 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:19.844022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.848021 (+ 3999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.848031 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:19.848096 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.848113 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:27:19.848207 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:27:19.848278 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":199,"threads_started":1}
I20260811 22:27:19.848662 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.835426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.24.71.195:32797)
0811 22:27:19.840087 (+ 4661us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:19.840099 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:19.840124 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:27:19.840199 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:19.840204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:19.840396 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:19.840559 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:27:19.841189 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.842373 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.846694 (+ 4321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.847654 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.847762 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.847945 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:19.848241 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 22:27:19.848340 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4544,"thread_start_us":125,"threads_started":1}
I20260811 22:27:19.849882 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:19.837209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50868 (local address 127.24.71.193:46391)
0811 22:27:19.841173 (+ 3964us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:19.841180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:19.841202 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:27:19.841269 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:19.841274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:19.841441 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:19.841872 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:27:19.842023 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.843229 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.844929 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:19.845738 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:19.847144 (+ 1406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:19.847297 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:19.847667 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 22:27:19.847772 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3859,"thread_start_us":118,"threads_started":1}
I20260811 22:27:19.851298 5113 tablet_service.cc:1467] Processing CreateTablet for tablet 7b25d30c9c9b4becbf59a2485538d086 (DEFAULT_TABLE table=test-table [id=96e44de1d1274cffb264c154404f2280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:27:19.851594 5263 tablet_service.cc:1467] Processing CreateTablet for tablet 7b25d30c9c9b4becbf59a2485538d086 (DEFAULT_TABLE table=test-table [id=96e44de1d1274cffb264c154404f2280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:27:19.852569 5113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b25d30c9c9b4becbf59a2485538d086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:27:19.852569 5263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b25d30c9c9b4becbf59a2485538d086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:27:19.852581 4964 tablet_service.cc:1467] Processing CreateTablet for tablet 7b25d30c9c9b4becbf59a2485538d086 (DEFAULT_TABLE table=test-table [id=96e44de1d1274cffb264c154404f2280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:27:19.853616 4964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b25d30c9c9b4becbf59a2485538d086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:27:19.863574 5358 tablet_bootstrap.cc:492] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a: Bootstrap starting.
I20260811 22:27:19.865134 5359 tablet_bootstrap.cc:492] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: Bootstrap starting.
I20260811 22:27:19.868117 5359 tablet_bootstrap.cc:654] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: Neither blocks nor log segments found. Creating new log.
I20260811 22:27:19.869072 5359 log.cc:824] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: Log is configured to *not* fsync() on all Append() calls
I20260811 22:27:19.869805 5358 tablet_bootstrap.cc:654] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a: Neither blocks nor log segments found. Creating new log.
I20260811 22:27:19.870651 5360 tablet_bootstrap.cc:492] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: Bootstrap starting.
I20260811 22:27:19.871078 5358 log.cc:824] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:27:19.871730 5359 tablet_bootstrap.cc:492] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: No bootstrap required, opened a new log
I20260811 22:27:19.871968 5359 ts_tablet_manager.cc:1397] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20260811 22:27:19.873338 5360 tablet_bootstrap.cc:654] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: Neither blocks nor log segments found. Creating new log.
I20260811 22:27:19.876524 5360 log.cc:824] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: Log is configured to *not* fsync() on all Append() calls
I20260811 22:27:19.876488 5359 raft_consensus.cc:348] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:19.876801 5359 raft_consensus.cc:374] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:27:19.876858 5359 raft_consensus.cc:729] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6946371edb40dda12cd6dd9aaa5992, State: Initialized, Role: FOLLOWER
I20260811 22:27:19.877555 5359 consensus_queue.cc:260] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:19.878651 5338 heartbeater.cc:503] Master 127.24.71.254:40281 was elected leader, sending a full tablet report...
I20260811 22:27:19.879088 5359 ts_tablet_manager.cc:1428] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:27:19.885962 5358 tablet_bootstrap.cc:492] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a: No bootstrap required, opened a new log
I20260811 22:27:19.886147 5360 tablet_bootstrap.cc:492] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: No bootstrap required, opened a new log
I20260811 22:27:19.886231 5358 ts_tablet_manager.cc:1397] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.000s
I20260811 22:27:19.886381 5360 ts_tablet_manager.cc:1397] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260811 22:27:19.893225 5358 raft_consensus.cc:348] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:19.893255 5360 raft_consensus.cc:348] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:19.893592 5360 raft_consensus.cc:374] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:27:19.893657 5360 raft_consensus.cc:729] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8074fd68d1e74c90ae366168740fd3ee, State: Initialized, Role: FOLLOWER
I20260811 22:27:19.893592 5358 raft_consensus.cc:374] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:27:19.893728 5358 raft_consensus.cc:729] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1bb4d55cba648c890ba2c1de267f54a, State: Initialized, Role: FOLLOWER
I20260811 22:27:19.894312 5360 consensus_queue.cc:260] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:19.894647 5358 consensus_queue.cc:260] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:19.895872 5358 ts_tablet_manager.cc:1428] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 22:27:19.896339 5186 heartbeater.cc:503] Master 127.24.71.254:40281 was elected leader, sending a full tablet report...
I20260811 22:27:19.895769 5360 ts_tablet_manager.cc:1428] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
W20260811 22:27:19.976555 5189 tablet.cc:2369] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:27:20.001354 5339 tablet.cc:2369] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:27:20.050742 5040 tablet.cc:2369] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:27:20.353645 5366 raft_consensus.cc:482] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:27:20.354040 5366 raft_consensus.cc:504] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:20.355245 5366 leader_election.cc:288] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba6946371edb40dda12cd6dd9aaa5992 (127.24.71.195:32797), a1bb4d55cba648c890ba2c1de267f54a (127.24.71.193:46391)
I20260811 22:27:20.357255 5364 raft_consensus.cc:482] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:27:20.357499 5364 raft_consensus.cc:504] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:20.359066 5364 leader_election.cc:288] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8074fd68d1e74c90ae366168740fd3ee (127.24.71.194:43955), a1bb4d55cba648c890ba2c1de267f54a (127.24.71.193:46391)
I20260811 22:27:20.361294 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:20.355725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:39301 (local address 127.24.71.193:46391)
0811 22:27:20.355898 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:20.355903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:20.356254 (+ 351us) server_negotiation.cc:407] Connection header received
0811 22:27:20.356350 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:20.356353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:20.356407 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:20.356543 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:27:20.356878 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.357700 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.359973 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.360585 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.360690 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:20.360733 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:20.361105 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 22:27:20.361189 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:27:20.361322 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:20.355612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:32797 (local address 127.24.71.194:56493)
0811 22:27:20.356176 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:20.356190 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:20.356260 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:20.356577 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:20.356581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:20.356606 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:27:20.356734 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:20.356744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.357875 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.357879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:20.359669 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:20.359684 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.360799 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.360803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:20.360929 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:20.360982 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 22:27:20.361058 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:27:20.361141 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":310,"thread_start_us":152,"threads_started":1}
I20260811 22:27:20.361792 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:20.355752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:56493 (local address 127.24.71.195:32797)
0811 22:27:20.356069 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:20.356073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:20.356274 (+ 201us) server_negotiation.cc:407] Connection header received
0811 22:27:20.356350 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:20.356353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:20.356407 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:20.356511 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:27:20.356893 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.357713 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.359873 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.360539 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.360675 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:20.360731 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:20.361563 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 22:27:20.361657 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
I20260811 22:27:20.362174 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:20.355612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:46391 (local address 127.24.71.194:39301)
0811 22:27:20.356100 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:20.356117 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:20.356260 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:20.356577 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:20.356581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:20.356682 (+ 101us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:27:20.356758 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:20.356763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.357887 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.357889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:20.359806 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:20.359816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.360799 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.360803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:20.360925 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:20.360971 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:27:20.361766 (+ 795us) client_negotiation.cc:240] Negotiation successful
0811 22:27:20.361996 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":312,"spinlock_wait_cycles":21376,"thread_start_us":88,"threads_started":1}
I20260811 22:27:20.362354 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b25d30c9c9b4becbf59a2485538d086" candidate_uuid: "8074fd68d1e74c90ae366168740fd3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6946371edb40dda12cd6dd9aaa5992" is_pre_election: true
I20260811 22:27:20.362617 5270 raft_consensus.cc:2454] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8074fd68d1e74c90ae366168740fd3ee in term 0.
I20260811 22:27:20.362841 4988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b25d30c9c9b4becbf59a2485538d086" candidate_uuid: "8074fd68d1e74c90ae366168740fd3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1bb4d55cba648c890ba2c1de267f54a" is_pre_election: true
I20260811 22:27:20.363157 4988 raft_consensus.cc:2454] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8074fd68d1e74c90ae366168740fd3ee in term 0.
I20260811 22:27:20.363154 5064 leader_election.cc:302] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8074fd68d1e74c90ae366168740fd3ee, ba6946371edb40dda12cd6dd9aaa5992; no voters:
I20260811 22:27:20.363461 5366 raft_consensus.cc:2781] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:27:20.363544 5366 raft_consensus.cc:482] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:27:20.363576 5366 raft_consensus.cc:3037] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:27:20.364880 5379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:20.359259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:43955 (local address 127.24.71.195:51847)
0811 22:27:20.360289 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:20.360307 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:20.360427 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:20.360836 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:20.360839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:20.360878 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:27:20.361002 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:20.361009 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.362156 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.362159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:20.363264 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:20.363272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.364339 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.364342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:20.364537 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:20.364565 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:27:20.364640 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:27:20.364708 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":36,"thread_start_us":84,"threads_started":1}
I20260811 22:27:20.364823 5366 raft_consensus.cc:504] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:20.365227 5366 leader_election.cc:288] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [CANDIDATE]: Term 1 election: Requested vote from peers ba6946371edb40dda12cd6dd9aaa5992 (127.24.71.195:32797), a1bb4d55cba648c890ba2c1de267f54a (127.24.71.193:46391)
I20260811 22:27:20.365566 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b25d30c9c9b4becbf59a2485538d086" candidate_uuid: "8074fd68d1e74c90ae366168740fd3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba6946371edb40dda12cd6dd9aaa5992"
I20260811 22:27:20.365692 5270 raft_consensus.cc:3037] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:27:20.365813 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:20.359379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:51847 (local address 127.24.71.194:43955)
0811 22:27:20.359810 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:20.359819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:20.360371 (+ 552us) server_negotiation.cc:407] Connection header received
0811 22:27:20.360544 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:20.360550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:20.360605 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:20.360688 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:27:20.361129 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.362025 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.363419 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.364109 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.364234 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:20.364281 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:20.365123 (+ 842us) server_negotiation.cc:299] Negotiation successful
0811 22:27:20.365217 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":89,"threads_started":1}
I20260811 22:27:20.365768 4988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b25d30c9c9b4becbf59a2485538d086" candidate_uuid: "8074fd68d1e74c90ae366168740fd3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1bb4d55cba648c890ba2c1de267f54a"
I20260811 22:27:20.365911 4988 raft_consensus.cc:3037] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:27:20.366238 5131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b25d30c9c9b4becbf59a2485538d086" candidate_uuid: "ba6946371edb40dda12cd6dd9aaa5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8074fd68d1e74c90ae366168740fd3ee" is_pre_election: true
I20260811 22:27:20.366482 5131 raft_consensus.cc:2379] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba6946371edb40dda12cd6dd9aaa5992 in current term 1: Already voted for candidate 8074fd68d1e74c90ae366168740fd3ee in this term.
I20260811 22:27:20.366632 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:20.359682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:46391 (local address 127.24.71.195:40421)
0811 22:27:20.360729 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:20.360742 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:27:20.360830 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:27:20.361228 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:27:20.361230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:27:20.361246 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:27:20.361395 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:20.361402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.363198 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.363202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:20.364857 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:27:20.364869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.366198 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.366202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:27:20.366306 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:20.366332 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:27:20.366413 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:27:20.366480 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":353,"thread_start_us":73,"threads_started":1}
I20260811 22:27:20.367040 5383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:20.359959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:40421 (local address 127.24.71.193:46391)
0811 22:27:20.360297 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:20.360307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:20.360909 (+ 602us) server_negotiation.cc:407] Connection header received
0811 22:27:20.360959 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:20.360963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:20.361010 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:20.361082 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:27:20.361531 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.363076 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.365209 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.365943 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.366076 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:20.366128 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:20.366787 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 22:27:20.366870 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260811 22:27:20.367116 5270 raft_consensus.cc:2454] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8074fd68d1e74c90ae366168740fd3ee in term 1.
I20260811 22:27:20.367390 4988 raft_consensus.cc:2454] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8074fd68d1e74c90ae366168740fd3ee in term 1.
I20260811 22:27:20.367518 5064 leader_election.cc:302] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8074fd68d1e74c90ae366168740fd3ee, ba6946371edb40dda12cd6dd9aaa5992; no voters:
I20260811 22:27:20.367882 5366 raft_consensus.cc:2781] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:27:20.367880 4988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b25d30c9c9b4becbf59a2485538d086" candidate_uuid: "ba6946371edb40dda12cd6dd9aaa5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1bb4d55cba648c890ba2c1de267f54a" is_pre_election: true
I20260811 22:27:20.368074 4988 raft_consensus.cc:2379] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba6946371edb40dda12cd6dd9aaa5992 in current term 1: Already voted for candidate 8074fd68d1e74c90ae366168740fd3ee in this term.
I20260811 22:27:20.368162 5366 raft_consensus.cc:686] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 1 LEADER]: Becoming Leader. State: Replica: 8074fd68d1e74c90ae366168740fd3ee, State: Running, Role: LEADER
I20260811 22:27:20.368523 5366 consensus_queue.cc:237] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } }
I20260811 22:27:20.368569 5214 leader_election.cc:302] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba6946371edb40dda12cd6dd9aaa5992; no voters: 8074fd68d1e74c90ae366168740fd3ee, a1bb4d55cba648c890ba2c1de267f54a
I20260811 22:27:20.368886 5364 raft_consensus.cc:2726] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 22:27:20.371771 4800 catalog_manager.cc:5495] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 8074fd68d1e74c90ae366168740fd3ee (127.24.71.194). New cstate: current_term: 1 leader_uuid: "8074fd68d1e74c90ae366168740fd3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8074fd68d1e74c90ae366168740fd3ee" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 } health_report { overall_health: UNKNOWN } } }
I20260811 22:27:20.410218 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:27:20.406187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.24.71.194:43955)
0811 22:27:20.406343 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:27:20.406354 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:27:20.406536 (+ 182us) server_negotiation.cc:407] Connection header received
0811 22:27:20.406719 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:27:20.406721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:27:20.406781 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:27:20.406884 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:27:20.407156 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.407670 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.408649 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:27:20.408989 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:27:20.409090 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:27:20.409386 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:27:20.409814 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:27:20.409914 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:27:20.409970 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 22:27:20.410035 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:27:20.416344 4988 raft_consensus.cc:1261] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a [term 1 FOLLOWER]: Refusing update from remote peer 8074fd68d1e74c90ae366168740fd3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:27:20.416384 5270 raft_consensus.cc:1261] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 1 FOLLOWER]: Refusing update from remote peer 8074fd68d1e74c90ae366168740fd3ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:27:20.417366 5386 consensus_queue.cc:1035] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1bb4d55cba648c890ba2c1de267f54a" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:27:20.417582 5366 consensus_queue.cc:1035] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba6946371edb40dda12cd6dd9aaa5992" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 32797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:27:20.427707 5398 mvcc.cc:205] Tried to move back new op lower bound from 7317451736738086912 to 7317451736552521728. Current Snapshot: MvccSnapshot[applied={T|T < 7317451736738086912}]
I20260811 22:27:20.434999 4801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:27:20.435205 4801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:27:20.438153 4801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d890dd6d9fc34250859534ad7e08fa83: Sending DeleteTablet for 3 replicas of tablet 7b25d30c9c9b4becbf59a2485538d086
I20260811 22:27:20.439218 4964 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b25d30c9c9b4becbf59a2485538d086 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:27:20 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:50868
I20260811 22:27:20.439215 5113 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b25d30c9c9b4becbf59a2485538d086 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:27:20 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:38706
I20260811 22:27:20.439774 5408 tablet_replica.cc:330] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: stopping tablet replica
I20260811 22:27:20.439747 5263 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b25d30c9c9b4becbf59a2485538d086 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:27:20 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:54490
I20260811 22:27:20.440007 5407 tablet_replica.cc:330] T 7b25d30c9c9b4becbf59a2485538d086 P a1bb4d55cba648c890ba2c1de267f54a: stopping tablet replica
I20260811 22:27:20.440162 5408 raft_consensus.cc:2229] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:27:20.440371 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 4893
I20260811 22:27:20.445123 5408 raft_consensus.cc:2258] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:27:20.446858 5408 ts_tablet_manager.cc:1905] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:27:20.450922 5408 ts_tablet_manager.cc:1918] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:27:20.451046 5408 log.cc:1196] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/7b25d30c9c9b4becbf59a2485538d086
I20260811 22:27:20.451377 5408 ts_tablet_manager.cc:1939] T 7b25d30c9c9b4becbf59a2485538d086 P 8074fd68d1e74c90ae366168740fd3ee: Deleting consensus metadata
I20260811 22:27:20.451380 5412 tablet_replica.cc:330] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: stopping tablet replica
I20260811 22:27:20.451917 5412 raft_consensus.cc:2229] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:27:20.452332 5412 raft_consensus.cc:2258] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 22:27:20.452878 4786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:46391 (error 108)
I20260811 22:27:20.453420 4786 catalog_manager.cc:4841] TS 8074fd68d1e74c90ae366168740fd3ee (127.24.71.194:43955): tablet 7b25d30c9c9b4becbf59a2485538d086 (table test-table [id=96e44de1d1274cffb264c154404f2280]) successfully deleted
I20260811 22:27:20.453824 5412 ts_tablet_manager.cc:1905] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:27:20.454330 5414 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:27:20.453764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:46391 (local address 127.0.0.1:50878)
0811 22:27:20.454102 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:27:20.454187 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:46391: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":105,"threads_started":1}
I20260811 22:27:20.454439 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 5046
W20260811 22:27:20.459093 4786 catalog_manager.cc:4584] TS a1bb4d55cba648c890ba2c1de267f54a (127.24.71.193:46391): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b25d30c9c9b4becbf59a2485538d086: Network error: Client connection negotiation failed: client connection to 127.24.71.193:46391: connect: Connection reset by peer (error 104)
I20260811 22:27:20.463104 5412 ts_tablet_manager.cc:1918] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:27:20.463236 5412 log.cc:1196] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/7b25d30c9c9b4becbf59a2485538d086
I20260811 22:27:20.463562 5412 ts_tablet_manager.cc:1939] T 7b25d30c9c9b4becbf59a2485538d086 P ba6946371edb40dda12cd6dd9aaa5992: Deleting consensus metadata
I20260811 22:27:20.465162 4785 catalog_manager.cc:4841] TS ba6946371edb40dda12cd6dd9aaa5992 (127.24.71.195:32797): tablet 7b25d30c9c9b4becbf59a2485538d086 (table test-table [id=96e44de1d1274cffb264c154404f2280]) successfully deleted
I20260811 22:27:20.467829 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 5194
I20260811 22:27:20.476653 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 4766
I20260811 22:27:20.486099 24863 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:27:20.831480 24863 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:27:20.831615 24863 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:27:21.176838 24863 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:27:21.177 UTC [4308] LOG: received smart shutdown request
2026-08-11 22:27:21.182 UTC [4308] LOG: background worker "logical replication launcher" (PID 4320) exited with exit code 1
2026-08-11 22:27:21.182 UTC [4314] LOG: shutting down
2026-08-11 22:27:21.362 UTC [4308] LOG: database system is shut down
2026-08-11T22:27:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51504 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:27:21 dist-test-slave-mvvj krb5kdc[5423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:27:21 dist-test-slave-mvvj krb5kdc[5423](info): set up 2 sockets
Aug 11 22:27:21 dist-test-slave-mvvj krb5kdc[5423](info): commencing operation
krb5kdc: starting...
W20260811 22:27:22.676556 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:27:22 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:27:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:27:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.71.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.71.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/rangeradmin_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/rangeradmin_127.24.71.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.71.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.71.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/rangerlookup_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/rangerlookup_127.24.71.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.71.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.71.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/HTTP_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/HTTP_127.24.71.212@KRBTEST.COM.keytab.
I20260811 22:27:22.880004 24863 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskapOnzC/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/postgres -l logfile start
2026-08-11 22:27:25.677 UTC [5455] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:27:25.677 UTC [5455] LOG: listening on IPv4 address "127.24.71.212", port 36907
2026-08-11 22:27:25.681 UTC [5455] LOG: listening on Unix socket "/tmp/.s.PGSQL.36907"
2026-08-11 22:27:25.722 UTC [5458] LOG: database system was shut down at 2026-08-11 22:27:23 UTC
2026-08-11 22:27:25.727 UTC [5455] LOG: database system is ready to accept connections
I20260811 22:27:26.731201 24863 mini_postgres.cc:96] Postgres bound to 36907
2026-08-11 22:27:26.739 UTC [5467] FATAL: database "slave" does not exist
127.24.71.212:36907 - accepting connections
I20260811 22:27:26.740412 24863 mini_ranger.cc:161] Starting Ranger...
I20260811 22:27:26.759598 24863 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:27:27.332211 24863 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:27:27.332329 24863 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-admin
2026-08-11 22:27:27,622 [I] DB FLAVOR :POSTGRES
2026-08-11 22:27:27,622 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:27:27,622 [I] Checking connection..
2026-08-11 22:27:27,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:27:27,886 [I] Checking connection passed.
2026-08-11 22:27:27,886 [I] --------- Verifying version history table ---------
2026-08-11 22:27:27,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:27:28,150 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:27:28,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:27:28,430 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:27:28,431 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:27:28,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:27:28.709 UTC [5540] WARNING: there is no transaction in progress
2026-08-11 22:27:28,718 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:27:28,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:27:29,005 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:27:29,005 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:27:29,005 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:27:29,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:27:29,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mvvj.c.gcp-upstream.internal','N') ;"
2026-08-11 22:27:29,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:27:30,059 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:27:30,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:27:30,341 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:27:30,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:27:30,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:27:30,867 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:27:30,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:27:31.975 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:31.997 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.018 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.037 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.061 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.086 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.155 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.163 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.180 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.192 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.204 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.221 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.233 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.242 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.252 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:32.261 UTC [5660] WARNING: there is no transaction in progress
2026-08-11 22:27:33,065 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:27:33,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:27:33,351 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:27:33,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:27:33,627 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:27:33,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:27:33,882 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:27:33,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:27:34,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:27:34,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mvvj.c.gcp-upstream.internal';"
2026-08-11 22:27:34,687 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:27:34,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:27:34,935 [I] DB_PATCHES have already been applied
I20260811 22:27:34.941727 24863 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-admin:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskapOnzC/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:27:34.941843 24863 mini_ranger.cc:193] Using host: 127.24.71.212
I20260811 22:27:34.946749 24863 mini_ranger.cc:227] Ranger admin URL: http://127.24.71.212:36643
Aug 11, 2026 10:27:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:27:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:27:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:27:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:27:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:27:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:27:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:27:36.158815 24863 mini_ranger.cc:160] Time spent starting Ranger: real 9.418s user 0.000s sys 0.013s
Aug 11, 2026 10:27:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:27:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:27:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:27:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:27:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:27:58.422434 24863 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.71.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.71.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/rangerkms_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.71.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/rangerkms_127.24.71.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.71.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.71.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.71.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/HTTP_127.24.71.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.71.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/HTTP_127.24.71.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:27:58 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487278, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:27:58.522331 24863 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:27:58.533850 24863 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:27:59.562480 24863 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:27:59.562563 24863 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-kms
I20260811 22:27:59.562583 24863 mini_ranger_kms.cc:227] Using postgres at 127.24.71.212:36907
2026-08-11 22:27:59,740 [I] DB FLAVOR :POSTGRES
2026-08-11 22:27:59,740 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:27:59,740 [I] Checking connection
2026-08-11 22:27:59,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:27:59,990 [I] connection success
2026-08-11 22:27:59,990 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:27:59,991 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:27:59,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:28:00,257 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:28:00,257 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:28:00,257 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:28:00,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.71.212:36907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:28:00,530 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:28:01.055989 24863 mini_ranger_kms.cc:310] Created kms service
I20260811 22:28:01.263187 24863 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:28:01.319211 24863 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:28:01 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487278, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:01.800452 24863 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:28:01.800614 24863 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-kms:/tmp/dist-test-taskapOnzC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskapOnzC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:28:01.800642 24863 mini_ranger_kms.cc:240] Using host: 127.24.71.212
I20260811 22:28:01.806425 24863 mini_ranger_kms.cc:276] Ranger KMS PID: 5862
I20260811 22:28:01.806526 24863 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.71.212:53453
Aug 11, 2026 10:28:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:28:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:28:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:28:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:28:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:28:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 22:28:04.293704 24863 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.771s user 0.000s sys 0.161s
I20260811 22:28:04.293850 24863 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:28:04.293938 24863 mini_ranger_kms.cc:401] 127.24.71.212:53453/kms/v1/keys
Aug 11 22:28:04 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487284, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.71.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 22:28:04 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487284, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.71.212@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
Aug 11 22:28:05 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487284, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.71.212@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
Aug 11 22:28:07 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:28:07.612021 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40165
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:40165
--ranger_config_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:53453/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:07.733145 5918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:07.733493 5918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:07.733536 5918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:07.738193 5918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:07.738286 5918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:07.738322 5918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:07.738339 5918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:07.738361 5918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:07.744522 5918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.71.212:53453/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:40165
--ranger_config_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:07.745723 5918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:07.746846 5918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:07.757771 5927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:07.757771 5925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:07.757789 5924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:07.757910 5918 server_base.cc:1034] running on GCE node
I20260811 22:28:07.758975 5918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:07.760490 5918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:07.761698 5918 hybrid_clock.cc:648] HybridClock initialized: now 1786487287761673 us; error 45 us; skew 500 ppm
Aug 11 22:28:07 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487287, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:07.770681 5918 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:28:07.772053 5918 webserver.cc:460] Webserver started at http://127.24.71.254:39421/ using document root <none> and password file <none>
I20260811 22:28:07.772732 5918 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:07.772830 5918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:07.773121 5918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:28:07 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487287, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:07.828632 5918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "5b0b8f3f46bf4a9e9a934032a769c18f"
format_stamp: "Formatted at 2026-08-11 22:28:07 on dist-test-slave-mvvj"
server_key: "249decced43e101d7423cd2203067cbe"
server_key_iv: "d3f0fba8d7e851c70cc6a65d54f18873"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:07.829360 5918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "5b0b8f3f46bf4a9e9a934032a769c18f"
format_stamp: "Formatted at 2026-08-11 22:28:07 on dist-test-slave-mvvj"
server_key: "249decced43e101d7423cd2203067cbe"
server_key_iv: "d3f0fba8d7e851c70cc6a65d54f18873"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:07.833913 5918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:28:07.850417 5936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:07.852272 5918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:07.852444 5918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "5b0b8f3f46bf4a9e9a934032a769c18f"
format_stamp: "Formatted at 2026-08-11 22:28:07 on dist-test-slave-mvvj"
server_key: "249decced43e101d7423cd2203067cbe"
server_key_iv: "d3f0fba8d7e851c70cc6a65d54f18873"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:07.852607 5918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:07.917071 5918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:07.920850 5918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:07.921092 5918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:07.930186 5918 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:40165
I20260811 22:28:07.930202 5994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:40165 every 8 connection(s)
I20260811 22:28:07.931236 5918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:28:07.932832 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 5918
I20260811 22:28:07.933008 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:28:07.933328 24863 external_mini_cluster.cc:1428] Setting key 0eb7c6e4fe143a375e09e708292c5694
I20260811 22:28:07.934710 5995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:07.941928 5995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Bootstrap starting.
Aug 11 22:28:07 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:07.944653 5995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:07.945765 5995 log.cc:824] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:07.948601 5995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: No bootstrap required, opened a new log
I20260811 22:28:07.951040 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:07.934869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.24.71.254:40165)
0811 22:28:07.935315 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:07.935335 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:07.935379 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:28:07.936354 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:07.936394 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:07.936849 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:07.937245 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:07.938211 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:07.939687 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:07.940840 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:07.941522 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:07.941700 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:07.944591 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:07.944603 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:07.944626 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:07.944668 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:07.947582 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:07.948179 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:07.948186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:07.948193 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:07.948333 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:07.948656 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:07.948660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:07.948662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:07.949197 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:07.949399 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:07.949722 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 22:28:07.950019 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":119,"threads_started":1}
I20260811 22:28:07.952718 5995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } }
I20260811 22:28:07.953002 5995 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:07.953078 5995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b0b8f3f46bf4a9e9a934032a769c18f, State: Initialized, Role: FOLLOWER
I20260811 22:28:07.953558 5995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } }
I20260811 22:28:07.953714 5995 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:28:07.953783 5995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:28:07.953893 5995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:07.954879 5995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } }
I20260811 22:28:07.955245 5995 leader_election.cc:302] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b0b8f3f46bf4a9e9a934032a769c18f; no voters:
I20260811 22:28:07.955581 5995 leader_election.cc:288] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:28:07.956012 6000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:28:07.956261 6000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 1 LEADER]: Becoming Leader. State: Replica: 5b0b8f3f46bf4a9e9a934032a769c18f, State: Running, Role: LEADER
I20260811 22:28:07.956578 6000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } }
I20260811 22:28:07.957024 5995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:28:07.958554 6001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } } }
I20260811 22:28:07.958595 6002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b0b8f3f46bf4a9e9a934032a769c18f. Latest consensus state: current_term: 1 leader_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } } }
I20260811 22:28:07.958814 6002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:07.958731 6001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:07.959585 6015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:28:07.959570 5995 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:28:07.965983 6015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:28:07.972821 6015 catalog_manager.cc:1347] Generated new cluster ID: e44c96e59f314f3f8ecb72cc8829d470
I20260811 22:28:07.972908 6015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:28:07.995218 6015 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:28:07.996742 6015 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:28:08.006052 6015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Generated new TSK 0
I20260811 22:28:08.007087 6015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:28:08.707010 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:53453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40165
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:08.834474 6047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:08.834805 6047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:08.834882 6047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:08.839749 6047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:08.839901 6047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:08.840060 6047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:28:08.846657 6047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.71.212:53453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:08.848026 6047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:08.849151 6047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:08.860957 6052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:08.861094 6053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:08.860957 6055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:08.862052 6047 server_base.cc:1034] running on GCE node
I20260811 22:28:08.862629 6047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:08.863427 6047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:08.864671 6047 hybrid_clock.cc:648] HybridClock initialized: now 1786487288864651 us; error 47 us; skew 500 ppm
Aug 11 22:28:08 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487288, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:08.867955 6047 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:28:08.869062 6047 webserver.cc:460] Webserver started at http://127.24.71.193:39463/ using document root <none> and password file <none>
I20260811 22:28:08.869675 6047 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:08.869741 6047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:08.869976 6047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:28:08 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487288, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:08.886814 6047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "b23ffb96562649b3a1371daea3efe43c"
format_stamp: "Formatted at 2026-08-11 22:28:08 on dist-test-slave-mvvj"
server_key: "8ccec435b161ce267603971d7626e982"
server_key_iv: "b328e021d259fb47facc8ab6afe0f55b"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:08.887527 6047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "b23ffb96562649b3a1371daea3efe43c"
format_stamp: "Formatted at 2026-08-11 22:28:08 on dist-test-slave-mvvj"
server_key: "8ccec435b161ce267603971d7626e982"
server_key_iv: "b328e021d259fb47facc8ab6afe0f55b"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:08.892159 6047 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:28:08.914522 6064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:08.916553 6047 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:28:08.916711 6047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "b23ffb96562649b3a1371daea3efe43c"
format_stamp: "Formatted at 2026-08-11 22:28:08 on dist-test-slave-mvvj"
server_key: "8ccec435b161ce267603971d7626e982"
server_key_iv: "b328e021d259fb47facc8ab6afe0f55b"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:08.916865 6047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:08.932511 6047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:08.936896 6047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:08.937232 6047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:08.938149 6047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:08.939694 6047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:08.939872 6047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 22:28:08.940028 6047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:08.940127 6047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:08.953626 6193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:42569 every 8 connection(s)
I20260811 22:28:08.953629 6047 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:42569
I20260811 22:28:08.954972 6047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:28:08.962008 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 6047
I20260811 22:28:08.962158 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:28:08.962553 24863 external_mini_cluster.cc:1428] Setting key a6e4ee1f9b4be40c5c29bd375c0cc3a8
Aug 11 22:28:08 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487288, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:08.972390 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:08.957183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:44687 (local address 127.24.71.254:40165)
0811 22:28:08.957476 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:08.957483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:08.958575 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 22:28:08.959746 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:08.959750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:08.959812 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:08.959927 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:08.961060 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:08.961638 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:08.962873 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:08.963217 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:08.963340 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:08.966608 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:08.966614 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:08.966619 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:08.966657 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:08.968829 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:08.969603 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:08.969607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:08.969609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:08.969711 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:08.970156 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:08.970159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:08.970161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:08.970540 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:08.970648 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:08.972037 (+ 1389us) server_negotiation.cc:299] Negotiation successful
0811 22:28:08.972191 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":128,"threads_started":1}
I20260811 22:28:08.973155 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:08.957767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40165 (local address 127.24.71.193:44687)
0811 22:28:08.958399 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:08.958462 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:08.959529 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:08.960119 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:08.960126 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:08.960409 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:08.960902 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:08.960916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:08.961780 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:08.961788 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:08.962744 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:08.962755 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:08.963399 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:08.963403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:08.963580 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:08.964453 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:08.964476 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:08.966359 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:08.968993 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:08.969001 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:08.969023 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:08.969466 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:08.969818 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:08.969821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:08.969823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:08.970038 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:08.970709 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:08.970717 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:08.971148 (+ 431us) client_negotiation.cc:769] Sending connection context
0811 22:28:08.971376 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 22:28:08.971780 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":141,"threads_started":1}
I20260811 22:28:08.974846 6194 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40165
I20260811 22:28:08.975152 6194 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:08.975857 6194 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
I20260811 22:28:08.977811 5953 ts_manager.cc:194] Registered new tserver with Master: b23ffb96562649b3a1371daea3efe43c (127.24.71.193:42569)
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:28:08.979406 5953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:44687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:28:09.016065 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:53453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40165
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:28:09.184363 6204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:09.184759 6204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:09.184828 6204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:09.190899 6204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:09.190979 6204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:09.191072 6204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:28:09.199139 6204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.71.212:53453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:09.200706 6204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:09.202080 6204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:09.215415 6213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:09.215780 6211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:09.215895 6210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:09.217067 6204 server_base.cc:1034] running on GCE node
I20260811 22:28:09.217634 6204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:09.218549 6204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:09.219765 6204 hybrid_clock.cc:648] HybridClock initialized: now 1786487289219743 us; error 36 us; skew 500 ppm
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:09.223645 6204 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:28:09.224907 6204 webserver.cc:460] Webserver started at http://127.24.71.194:45427/ using document root <none> and password file <none>
I20260811 22:28:09.225792 6204 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:09.225891 6204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:09.226295 6204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:09.244279 6204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "eab8daf8346f488b93b435279054faad"
format_stamp: "Formatted at 2026-08-11 22:28:09 on dist-test-slave-mvvj"
server_key: "8c6c01c3284e29a8f69d6a3acabf6562"
server_key_iv: "a966fc43d288712ede498dd523ae4dcb"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:09.245142 6204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "eab8daf8346f488b93b435279054faad"
format_stamp: "Formatted at 2026-08-11 22:28:09 on dist-test-slave-mvvj"
server_key: "8c6c01c3284e29a8f69d6a3acabf6562"
server_key_iv: "a966fc43d288712ede498dd523ae4dcb"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:09.251147 6204 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:28:09.272205 6220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:09.274004 6204 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 22:28:09.274211 6204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "eab8daf8346f488b93b435279054faad"
format_stamp: "Formatted at 2026-08-11 22:28:09 on dist-test-slave-mvvj"
server_key: "8c6c01c3284e29a8f69d6a3acabf6562"
server_key_iv: "a966fc43d288712ede498dd523ae4dcb"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:09.274387 6204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:09.291615 6204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:09.295569 6204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:09.295792 6204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:09.296612 6204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:09.297878 6204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:09.297966 6204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:09.298046 6204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:09.298080 6204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:09.313603 6204 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:42255
I20260811 22:28:09.314934 6204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:28:09.320698 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 6204
I20260811 22:28:09.320845 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:28:09.321213 24863 external_mini_cluster.cc:1428] Setting key a6462be902640382dcb74010e0954f48
I20260811 22:28:09.316706 6344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:42255 every 8 connection(s)
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:28:09.352047 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:09.319178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40165 (local address 127.24.71.194:33399)
0811 22:28:09.326075 (+ 6897us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:09.326151 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:09.327300 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:09.328131 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:09.328141 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:09.328525 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:09.329249 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:09.329269 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:09.330484 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:09.330487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:09.331864 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:09.331876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:09.334906 (+ 3030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:09.334909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:09.335132 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:09.341095 (+ 5963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:09.341123 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:09.345516 (+ 4393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:09.348311 (+ 2795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:09.348321 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:09.348344 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:09.348762 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:09.349173 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:09.349177 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:09.349179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:09.349396 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:09.349931 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:09.349939 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:09.350306 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 22:28:09.350519 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 22:28:09.350890 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3784,"thread_start_us":156,"threads_started":1}
I20260811 22:28:09.353212 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:09.321784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:33399 (local address 127.24.71.254:40165)
0811 22:28:09.321950 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:09.321956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:09.326261 (+ 4305us) server_negotiation.cc:407] Connection header received
0811 22:28:09.327594 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:09.327598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:09.327657 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:09.327750 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:09.329436 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:09.330303 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:09.334228 (+ 3925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:09.334592 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:09.334677 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:09.345855 (+ 11178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:09.345860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:09.345863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:09.345904 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:09.348001 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:09.348921 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:09.348926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:09.348928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:09.349044 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:09.349520 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:09.349524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:09.349526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:09.349775 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:09.349871 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:09.352847 (+ 2976us) server_negotiation.cc:299] Negotiation successful
0811 22:28:09.353008 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:28:09.354403 6345 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40165
I20260811 22:28:09.354682 6345 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:09.355347 6345 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
I20260811 22:28:09.356937 5953 ts_manager.cc:194] Registered new tserver with Master: eab8daf8346f488b93b435279054faad (127.24.71.194:42255)
I20260811 22:28:09.357755 5953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:33399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:28:09.372316 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:53453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40165
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:09.523039 6350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:09.523430 6350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:09.523510 6350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:09.528147 6350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:09.528241 6350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:09.528350 6350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:28:09.534107 6350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.71.212:53453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:09.535344 6350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:09.536512 6350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:28:09.547967 6350 server_base.cc:1034] running on GCE node
W20260811 22:28:09.548218 6356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:09.548256 6357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:09.548336 6359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:09.548967 6350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:09.549780 6350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:09.553213 6350 hybrid_clock.cc:648] HybridClock initialized: now 1786487289553162 us; error 49 us; skew 500 ppm
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:09.557286 6350 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:28:09.558597 6350 webserver.cc:460] Webserver started at http://127.24.71.195:39095/ using document root <none> and password file <none>
I20260811 22:28:09.559458 6350 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:09.559623 6350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:09.559969 6350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:09.584611 6350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "5deb6b047da5465182048a3aaf0fd053"
format_stamp: "Formatted at 2026-08-11 22:28:09 on dist-test-slave-mvvj"
server_key: "685a73cc7eb334f6e4a77ce5e1c32d84"
server_key_iv: "011bd077dca981a9fb232610f7feda47"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:09.585412 6350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "5deb6b047da5465182048a3aaf0fd053"
format_stamp: "Formatted at 2026-08-11 22:28:09 on dist-test-slave-mvvj"
server_key: "685a73cc7eb334f6e4a77ce5e1c32d84"
server_key_iv: "011bd077dca981a9fb232610f7feda47"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:09.595098 6350 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.000s
I20260811 22:28:09.619804 6367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:09.621795 6350 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260811 22:28:09.621980 6350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "5deb6b047da5465182048a3aaf0fd053"
format_stamp: "Formatted at 2026-08-11 22:28:09 on dist-test-slave-mvvj"
server_key: "685a73cc7eb334f6e4a77ce5e1c32d84"
server_key_iv: "011bd077dca981a9fb232610f7feda47"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:09.622165 6350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:09.643225 6350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:09.647145 6350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:09.647364 6350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:09.648114 6350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:09.649235 6350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:09.649309 6350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:09.649374 6350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:09.649395 6350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:09.664342 6350 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:41373
I20260811 22:28:09.665695 6350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:28:09.666905 6490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:41373 every 8 connection(s)
I20260811 22:28:09.675544 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 6350
I20260811 22:28:09.675685 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:28:09.676067 24863 external_mini_cluster.cc:1428] Setting key 427059e654991edcce8d56cfcbe907ae
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:09.690995 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:09.671546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:47871 (local address 127.24.71.254:40165)
0811 22:28:09.671685 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:09.671690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:09.672281 (+ 591us) server_negotiation.cc:407] Connection header received
0811 22:28:09.673611 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:09.673615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:09.673689 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:09.673749 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:09.675546 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:09.676424 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:09.680579 (+ 4155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:09.680929 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:09.681016 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:09.686312 (+ 5296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:09.686317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:09.686320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:09.686356 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:09.688177 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:09.689070 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:09.689074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:09.689076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:09.689177 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:09.689618 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:09.689622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:09.689624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:09.689868 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:09.689959 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:09.690690 (+ 731us) server_negotiation.cc:299] Negotiation successful
0811 22:28:09.690825 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:28:09.692978 6493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:09.670274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40165 (local address 127.24.71.195:47871)
0811 22:28:09.672094 (+ 1820us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:09.672154 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:09.673313 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:09.674203 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:09.674213 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:09.674614 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:09.675309 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:09.675330 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:09.678845 (+ 3515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:09.678849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:09.680411 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:09.680425 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:09.681257 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:09.681261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:09.681497 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:09.684087 (+ 2590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:09.684113 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:09.685990 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:09.688337 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:09.688345 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:09.688371 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:09.688916 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:09.689286 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:09.689290 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:09.689292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:09.689499 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:09.690031 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:09.690039 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:09.690405 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 22:28:09.691402 (+ 997us) client_negotiation.cc:240] Negotiation successful
0811 22:28:09.691793 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1581,"thread_start_us":170,"threads_started":1}
I20260811 22:28:09.694702 6491 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40165
I20260811 22:28:09.695015 6491 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:09.695684 6491 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
I20260811 22:28:09.697027 5953 ts_manager.cc:194] Registered new tserver with Master: 5deb6b047da5465182048a3aaf0fd053 (127.24.71.195:41373)
I20260811 22:28:09.697810 5953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:47871
I20260811 22:28:09.703006 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:28:09.703918 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 6047
I20260811 22:28:09.718444 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 6204
I20260811 22:28:09.726768 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 6350
I20260811 22:28:09.736629 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 5918
I20260811 22:28:09.747130 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40165
--webserver_interface=127.24.71.254
--webserver_port=39421
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:40165
--ranger_config_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:53453/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.71.212:53453/kms with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:09.865303 6494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:09.865687 6494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:09.865769 6494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:09.870080 6494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:09.870193 6494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:09.870232 6494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:09.870281 6494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:09.870314 6494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:09.876196 6494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.71.212:53453/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:40165
--ranger_config_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=39421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:09.877458 6494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:09.878597 6494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:09.888063 6503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:09.888046 6500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:09.888052 6501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:09.888468 6494 server_base.cc:1034] running on GCE node
I20260811 22:28:09.889130 6494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:09.890408 6494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:09.891631 6494 hybrid_clock.cc:648] HybridClock initialized: now 1786487289891613 us; error 32 us; skew 500 ppm
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:09.894853 6494 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:28:09.895795 6494 webserver.cc:460] Webserver started at http://127.24.71.254:39421/ using document root <none> and password file <none>
I20260811 22:28:09.896423 6494 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:09.896500 6494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:28:09 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487289, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:09.977093 6494 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260811 22:28:09.980048 6513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:09.981705 6494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:28:09.981871 6494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "5b0b8f3f46bf4a9e9a934032a769c18f"
format_stamp: "Formatted at 2026-08-11 22:28:07 on dist-test-slave-mvvj"
server_key: "249decced43e101d7423cd2203067cbe"
server_key_iv: "d3f0fba8d7e851c70cc6a65d54f18873"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:09.982493 6494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:10.013218 6494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:10.016973 6494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:10.017465 6494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:10.026932 6494 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:40165
I20260811 22:28:10.026986 6568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:40165 every 8 connection(s)
I20260811 22:28:10.028596 6494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:28:10.033830 6569 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:28:10.035753 6569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Bootstrap starting.
I20260811 22:28:10.037345 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 6494
I20260811 22:28:10.037873 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:53453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:42569
--local_ip_for_outbound_sockets=127.24.71.193
--tserver_master_addrs=127.24.71.254:40165
--webserver_port=39463
--webserver_interface=127.24.71.193
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.71.212:53453/kms with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:28:10.054227 6569 log.cc:824] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:10.062343 6569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:28:10.062875 6569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Bootstrap complete.
I20260811 22:28:10.067902 6569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } }
I20260811 22:28:10.071043 6569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b0b8f3f46bf4a9e9a934032a769c18f, State: Initialized, Role: FOLLOWER
I20260811 22:28:10.072256 6569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } }
I20260811 22:28:10.072544 6569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:28:10.072647 6569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:28:10.072784 6569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:28:10.075332 6569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } }
I20260811 22:28:10.075814 6569 leader_election.cc:302] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b0b8f3f46bf4a9e9a934032a769c18f; no voters:
I20260811 22:28:10.076269 6569 leader_election.cc:288] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:28:10.076516 6573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:28:10.076895 6573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [term 2 LEADER]: Becoming Leader. State: Replica: 5b0b8f3f46bf4a9e9a934032a769c18f, State: Running, Role: LEADER
I20260811 22:28:10.077394 6573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } }
I20260811 22:28:10.077770 6569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:28:10.079542 6576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b0b8f3f46bf4a9e9a934032a769c18f. Latest consensus state: current_term: 2 leader_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } } }
I20260811 22:28:10.079691 6576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:10.080808 6575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b0b8f3f46bf4a9e9a934032a769c18f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40165 } } }
I20260811 22:28:10.080935 6575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:10.081049 6584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:28:10.081120 6569 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:28:10.088340 6584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:28:10.089152 6584 catalog_manager.cc:1259] Loaded cluster ID: e44c96e59f314f3f8ecb72cc8829d470
I20260811 22:28:10.089215 6584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:28:10.092283 6584 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:28:10.093201 6584 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Loaded TSK: 0
I20260811 22:28:10.095050 6584 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:28:10.206270 6571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:10.206668 6571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:10.206737 6571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:10.213122 6571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:10.213217 6571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:10.213307 6571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:28:10.222153 6571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.71.212:53453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:42569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=39463
--tserver_master_addrs=127.24.71.254:40165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:10.223582 6571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:10.225061 6571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:10.239917 6611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:10.240475 6610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:10.241462 6613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:10.241981 6571 server_base.cc:1034] running on GCE node
I20260811 22:28:10.242571 6571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:10.243492 6571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:10.245414 6571 hybrid_clock.cc:648] HybridClock initialized: now 1786487290245381 us; error 67 us; skew 500 ppm
Aug 11 22:28:10 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487290, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:10.249891 6571 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:28:10.251261 6571 webserver.cc:460] Webserver started at http://127.24.71.193:39463/ using document root <none> and password file <none>
I20260811 22:28:10.252177 6571 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:10.252312 6571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:28:10 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487290, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:10.302594 6571 fs_manager.cc:709] Time spent opening directory manager: real 0.023s user 0.003s sys 0.000s
I20260811 22:28:10.308401 6622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:10.310370 6571 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260811 22:28:10.310556 6571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "b23ffb96562649b3a1371daea3efe43c"
format_stamp: "Formatted at 2026-08-11 22:28:08 on dist-test-slave-mvvj"
server_key: "8ccec435b161ce267603971d7626e982"
server_key_iv: "b328e021d259fb47facc8ab6afe0f55b"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:10.311398 6571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:10.355494 6571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:10.359901 6571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:10.360183 6571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:10.361085 6571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:10.362339 6571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:10.362425 6571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:10.362596 6571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:10.362640 6571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:10.376221 6571 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:42569
I20260811 22:28:10.376489 6745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:42569 every 8 connection(s)
I20260811 22:28:10.377422 6571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:28:10.378618 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 6571
I20260811 22:28:10.379149 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:53453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:42255
--local_ip_for_outbound_sockets=127.24.71.194
--tserver_master_addrs=127.24.71.254:40165
--webserver_port=45427
--webserver_interface=127.24.71.194
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.71.212:53453/kms with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:28:10 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487290, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:10.404445 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:10.381899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:43253 (local address 127.24.71.254:40165)
0811 22:28:10.382514 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:10.382536 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:10.382585 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:28:10.383880 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:10.383906 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:10.384355 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:10.384735 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:10.386627 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.388274 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.390105 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.390859 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.391040 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:10.396402 (+ 5362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:10.396411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:10.396434 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:10.396495 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:10.398947 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:10.399864 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:10.399876 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:10.399886 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:10.400077 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:10.400617 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:10.400623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:10.400627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:10.401265 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:10.401428 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:10.403083 (+ 1655us) server_negotiation.cc:299] Negotiation successful
0811 22:28:10.403385 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":123,"threads_started":1}
I20260811 22:28:10.405933 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:10.380229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40165 (local address 127.24.71.193:43253)
0811 22:28:10.381927 (+ 1698us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:10.381995 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:10.383179 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:10.385251 (+ 2072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:10.385262 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:10.385673 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:10.386413 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:10.386439 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.388455 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.388458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:10.389945 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:10.389958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.391291 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.391295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:10.391487 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:10.393975 (+ 2488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:10.394002 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:10.396064 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:10.399127 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:10.399138 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:10.399165 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:10.399638 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:10.400223 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:10.400228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:10.400230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:10.400472 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:10.401509 (+ 1037us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:10.401520 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:10.401881 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 22:28:10.402101 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 22:28:10.402479 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1450,"thread_start_us":148,"threads_started":1}
I20260811 22:28:10.409051 6746 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40165
I20260811 22:28:10.409343 6746 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:10.410027 6746 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
W20260811 22:28:10.412573 6746 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:40165 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:28:10.412714 6746 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:10.412918 6746 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
I20260811 22:28:10.414049 6746 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:10.414243 6746 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
W20260811 22:28:10.415102 6746 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:28:10.549996 6749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:10.550447 6749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:10.550542 6749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:10.557847 6749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:10.557960 6749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:10.558084 6749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:28:10.567889 6749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.71.212:53453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:42255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=45427
--tserver_master_addrs=127.24.71.254:40165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:10.569455 6749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:10.570977 6749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:10.585562 6756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:10.585999 6759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:10.585623 6755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:10.586831 6749 server_base.cc:1034] running on GCE node
I20260811 22:28:10.587404 6749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:10.588380 6749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:10.589608 6749 hybrid_clock.cc:648] HybridClock initialized: now 1786487290589526 us; error 96 us; skew 500 ppm
Aug 11 22:28:10 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487290, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:10.593466 6749 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:28:10.594708 6749 webserver.cc:460] Webserver started at http://127.24.71.194:45427/ using document root <none> and password file <none>
I20260811 22:28:10.595539 6749 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:10.595664 6749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:28:10 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487290, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:10.619311 6749 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:28:10.622378 6769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:10.623931 6749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:10.624070 6749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "eab8daf8346f488b93b435279054faad"
format_stamp: "Formatted at 2026-08-11 22:28:09 on dist-test-slave-mvvj"
server_key: "8c6c01c3284e29a8f69d6a3acabf6562"
server_key_iv: "a966fc43d288712ede498dd523ae4dcb"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:10.624675 6749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:10.667210 6749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:10.670749 6749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:10.670975 6749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:10.671784 6749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:10.672869 6749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:10.672924 6749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:10.672993 6749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:10.673026 6749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:10.684438 6749 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:42255
I20260811 22:28:10.684746 6891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:42255 every 8 connection(s)
I20260811 22:28:10.685662 6749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:28:10.690523 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 6749
I20260811 22:28:10.691069 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.71.212:53453/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:41373
--local_ip_for_outbound_sockets=127.24.71.195
--tserver_master_addrs=127.24.71.254:40165
--webserver_port=39095
--webserver_interface=127.24.71.195
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.71.212:53453/kms with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:28:10 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487290, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:10.707978 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:10.688165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:52397 (local address 127.24.71.254:40165)
0811 22:28:10.688336 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:10.688340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:10.690199 (+ 1859us) server_negotiation.cc:407] Connection header received
0811 22:28:10.691595 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:10.691600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:10.691675 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:10.691800 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:10.693454 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.694008 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.695688 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.696155 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.696297 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:10.702477 (+ 6180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:10.702483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:10.702486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:10.702526 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:10.704609 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:10.706123 (+ 1514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:10.706127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:10.706129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:10.706235 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:10.706663 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:10.706667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:10.706668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:10.706892 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:10.707013 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:10.707584 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 22:28:10.707737 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:28:10.709059 6896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:10.688694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40165 (local address 127.24.71.194:52397)
0811 22:28:10.689980 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:10.690055 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:10.691289 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:10.692083 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:10.692096 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:10.692501 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:10.693262 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:10.693280 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.694293 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.694298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:10.695507 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:10.695520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.696386 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.696391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:10.696642 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:10.700162 (+ 3520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:10.700190 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:10.702177 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:10.705479 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:10.705488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:10.705511 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:10.705978 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:10.706348 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:10.706352 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:10.706354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:10.706545 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:10.707014 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:10.707023 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:10.707323 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 22:28:10.707539 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 22:28:10.707959 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":149,"threads_started":1}
I20260811 22:28:10.710386 6892 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40165
I20260811 22:28:10.710692 6892 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:10.711444 6892 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
W20260811 22:28:10.713346 6892 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:40165 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:28:10.713586 6892 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:10.713841 6892 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
I20260811 22:28:10.714892 6892 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:10.715157 6892 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
W20260811 22:28:10.716264 6892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:28:10.853852 6897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:10.854183 6897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:10.854266 6897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:10.860422 6897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:10.860592 6897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:10.860729 6897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:28:10.869138 6897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.71.212:53453/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:41373
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=39095
--tserver_master_addrs=127.24.71.254:40165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:10.870890 6897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:10.872417 6897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:10.884382 6903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:10.884419 6905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:10.884377 6902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:10.884958 6897 server_base.cc:1034] running on GCE node
I20260811 22:28:10.885462 6897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:10.886206 6897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:10.887470 6897 hybrid_clock.cc:648] HybridClock initialized: now 1786487290887424 us; error 72 us; skew 500 ppm
Aug 11 22:28:10 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487290, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:10.890621 6897 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:28:10.891763 6897 webserver.cc:460] Webserver started at http://127.24.71.195:39095/ using document root <none> and password file <none>
I20260811 22:28:10.892402 6897 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:10.892514 6897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:28:10 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487290, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:10.922019 6897 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:28:10.925282 6922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:10.927099 6897 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:28:10.927275 6897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "5deb6b047da5465182048a3aaf0fd053"
format_stamp: "Formatted at 2026-08-11 22:28:09 on dist-test-slave-mvvj"
server_key: "685a73cc7eb334f6e4a77ce5e1c32d84"
server_key_iv: "011bd077dca981a9fb232610f7feda47"
server_key_version: "kuduclusterkey@0"
I20260811 22:28:10.928114 6897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:10.954972 6897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:10.958406 6897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:10.958644 6897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:10.959362 6897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:10.960441 6897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:10.960517 6897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:10.960588 6897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:10.960640 6897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:10.972507 6897 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:41373
I20260811 22:28:10.972673 7046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:41373 every 8 connection(s)
I20260811 22:28:10.974080 6897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:28:10.982339 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 6897
Aug 11 22:28:10 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487290, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:10.993273 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:10.978046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:53365 (local address 127.24.71.254:40165)
0811 22:28:10.978207 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:10.978212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:10.978232 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:28:10.979118 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:10.979122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:10.979208 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:10.979329 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:10.981023 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.981853 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.983969 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.984325 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.984413 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:10.988563 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:10.988570 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:10.988575 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:10.988606 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:10.990363 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:10.991114 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:10.991119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:10.991122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:10.991241 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:10.991743 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:10.991750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:10.991753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:10.992057 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:10.992149 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:10.992958 (+ 809us) server_negotiation.cc:299] Negotiation successful
0811 22:28:10.993097 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63,"spinlock_wait_cycles":2304}
I20260811 22:28:10.994725 7051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:10.983561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47554 (local address 127.24.71.254:40165)
0811 22:28:10.984153 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:10.984162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:10.984174 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:28:10.984215 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:10.984218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:10.984268 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:10.985317 (+ 1049us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:10.985428 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.986267 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.987575 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.987961 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.988074 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:10.990309 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:10.990315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:10.990318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:10.990357 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:10.992145 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:10.992661 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:10.992664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:10.992666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:10.992749 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:10.993095 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:10.993098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:10.993099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:10.993280 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:10.994361 (+ 1081us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:10.994454 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 22:28:10.994573 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"spinlock_wait_cycles":3456,"thread_start_us":117,"threads_started":1}
I20260811 22:28:10.995204 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:10.976949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40165 (local address 127.24.71.195:53365)
0811 22:28:10.977594 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:10.977657 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:10.978830 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:10.979574 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:10.979585 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:10.980018 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:10.980827 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:10.980850 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.982044 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.982048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:10.983762 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:10.983774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:10.984629 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:10.984632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:10.984841 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:10.985575 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:10.985598 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:10.987961 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:10.990495 (+ 2534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:10.990502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:10.990526 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:10.990983 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:10.991373 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:10.991377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:10.991381 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:10.991617 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:10.992227 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:10.992238 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:10.992644 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 22:28:10.993660 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0811 22:28:10.994052 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":157,"threads_started":1}
I20260811 22:28:10.996735 7047 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40165
I20260811 22:28:10.997025 7047 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:10.997570 7047 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
I20260811 22:28:10.999019 6531 ts_manager.cc:194] Registered new tserver with Master: 5deb6b047da5465182048a3aaf0fd053 (127.24.71.195:41373)
I20260811 22:28:11.000325 6531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:53365
Aug 11 22:28:11 dist-test-slave-mvvj krb5kdc[5423](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786487291, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:11.415366 6746 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:11.415696 6746 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
I20260811 22:28:11.416672 6531 ts_manager.cc:194] Registered new tserver with Master: b23ffb96562649b3a1371daea3efe43c (127.24.71.193:42569)
I20260811 22:28:11.417599 6531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:43253
Aug 11 22:28:11 dist-test-slave-mvvj krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487291, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for HTTP/127.24.71.212@KRBTEST.COM
I20260811 22:28:11.716560 6892 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:11.716900 6892 heartbeater.cc:511] Master 127.24.71.254:40165 requested a full tablet report, sending...
I20260811 22:28:11.717715 6531 ts_manager.cc:194] Registered new tserver with Master: eab8daf8346f488b93b435279054faad (127.24.71.194:42255)
I20260811 22:28:11.718544 6531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:52397
I20260811 22:28:11.728360 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:28:11.740692 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.730198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47562 (local address 127.24.71.254:40165)
0811 22:28:11.730848 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:11.730852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:11.730866 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:28:11.730909 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:11.730912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:11.730961 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:11.731057 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:11.732025 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.732817 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.734127 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.734446 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.734536 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.736444 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:11.736448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:11.736451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:11.736479 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:11.738398 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:11.738935 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:11.738939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:11.738942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:11.739037 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:11.739337 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:11.739341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:11.739343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:11.739571 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:11.739698 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:11.740379 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 22:28:11.740508 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":119,"threads_started":1}
I20260811 22:28:11.745335 6531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:28:11.750725 6531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:28:11.773308 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.763925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:41373 (local address 127.0.0.1:54762)
0811 22:28:11.766358 (+ 2433us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:11.766425 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:11.766603 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:11.767334 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:11.767342 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:11.767370 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:11.767527 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.767535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.769039 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.769043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.771379 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.771392 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.772737 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.772741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.772872 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.772915 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:28:11.773027 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:28:11.773150 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2220,"thread_start_us":132,"threads_started":1}
I20260811 22:28:11.774857 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.764063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54762 (local address 127.24.71.195:41373)
0811 22:28:11.764502 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:11.764510 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:11.766767 (+ 2257us) server_negotiation.cc:407] Connection header received
0811 22:28:11.766863 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:11.766869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:11.767134 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:11.767323 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:11.767673 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.768895 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.771535 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.772417 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.772520 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.772674 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:11.774577 (+ 1903us) server_negotiation.cc:299] Negotiation successful
0811 22:28:11.774699 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":132,"threads_started":1}
I20260811 22:28:11.778678 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.765611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:42255 (local address 127.0.0.1:41064)
0811 22:28:11.767976 (+ 2365us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:11.767990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:11.768075 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:11.769570 (+ 1495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:11.769573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:11.769594 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:11.769708 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.769715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.773267 (+ 3552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.773271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.774444 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.774465 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.778172 (+ 3707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.778178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.778360 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.778384 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:28:11.778462 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:28:11.778536 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2301,"mutex_wait_us":22,"thread_start_us":92,"threads_started":1}
I20260811 22:28:11.779187 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.765338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:42569 (local address 127.0.0.1:55620)
0811 22:28:11.767625 (+ 2287us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:11.767642 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:11.767739 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:11.771319 (+ 3580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:11.771323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:11.771343 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:11.771435 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.771442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.775645 (+ 4203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.775649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.776747 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.776759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.777937 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.777941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.778038 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.778066 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:28:11.778179 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:28:11.778280 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2215,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260811 22:28:11.778841 6976 tablet_service.cc:1467] Processing CreateTablet for tablet 559092d5fa7746b39766b19fe91a3a7a (DEFAULT_TABLE table=test-table [id=ed18ecca73ac438daa839b21be86b6d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:11.779750 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.765788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55620 (local address 127.24.71.193:42569)
0811 22:28:11.768369 (+ 2581us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:11.768377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:11.768404 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:28:11.768502 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:11.768511 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:11.768748 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:11.771621 (+ 2873us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:11.771797 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.773108 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.776921 (+ 3813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.777790 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.777910 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.778065 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:11.779501 (+ 1436us) server_negotiation.cc:299] Negotiation successful
0811 22:28:11.779615 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2453,"thread_start_us":88,"threads_started":1}
I20260811 22:28:11.780210 6976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 559092d5fa7746b39766b19fe91a3a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:11.780468 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.765788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.24.71.194:42255)
0811 22:28:11.769067 (+ 3279us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:11.769074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:11.769101 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:28:11.769178 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:11.769183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:11.769388 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:11.769882 (+ 494us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:11.770029 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.771156 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.774620 (+ 3464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.775493 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.779745 (+ 4252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.780074 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:11.780132 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 22:28:11.780324 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3156,"thread_start_us":86,"threads_started":1}
I20260811 22:28:11.782852 6665 tablet_service.cc:1467] Processing CreateTablet for tablet 559092d5fa7746b39766b19fe91a3a7a (DEFAULT_TABLE table=test-table [id=ed18ecca73ac438daa839b21be86b6d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:11.783275 6818 tablet_service.cc:1467] Processing CreateTablet for tablet 559092d5fa7746b39766b19fe91a3a7a (DEFAULT_TABLE table=test-table [id=ed18ecca73ac438daa839b21be86b6d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:11.784018 6665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 559092d5fa7746b39766b19fe91a3a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:11.784265 6818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 559092d5fa7746b39766b19fe91a3a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:11.793543 7074 tablet_bootstrap.cc:492] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: Bootstrap starting.
I20260811 22:28:11.796028 7074 tablet_bootstrap.cc:654] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:11.796377 7075 tablet_bootstrap.cc:492] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c: Bootstrap starting.
I20260811 22:28:11.797185 7074 log.cc:824] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:11.798834 7075 tablet_bootstrap.cc:654] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:11.799925 7073 tablet_bootstrap.cc:492] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: Bootstrap starting.
I20260811 22:28:11.800125 7074 tablet_bootstrap.cc:492] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: No bootstrap required, opened a new log
I20260811 22:28:11.800334 7074 ts_tablet_manager.cc:1397] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260811 22:28:11.802412 7073 tablet_bootstrap.cc:654] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:11.803345 7073 log.cc:824] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:11.804556 7075 log.cc:824] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:11.804589 7074 raft_consensus.cc:348] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } }
I20260811 22:28:11.805208 7074 raft_consensus.cc:374] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:11.805327 7074 raft_consensus.cc:729] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab8daf8346f488b93b435279054faad, State: Initialized, Role: FOLLOWER
I20260811 22:28:11.806051 7074 consensus_queue.cc:260] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } }
I20260811 22:28:11.807309 7073 tablet_bootstrap.cc:492] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: No bootstrap required, opened a new log
I20260811 22:28:11.807436 7074 ts_tablet_manager.cc:1428] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260811 22:28:11.807541 7073 ts_tablet_manager.cc:1397] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 22:28:11.807941 6892 heartbeater.cc:503] Master 127.24.71.254:40165 was elected leader, sending a full tablet report...
I20260811 22:28:11.812094 7073 raft_consensus.cc:348] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } }
I20260811 22:28:11.812407 7073 raft_consensus.cc:374] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:11.812466 7073 raft_consensus.cc:729] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5deb6b047da5465182048a3aaf0fd053, State: Initialized, Role: FOLLOWER
I20260811 22:28:11.813001 7073 consensus_queue.cc:260] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } }
I20260811 22:28:11.814163 7073 ts_tablet_manager.cc:1428] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:28:11.814194 7047 heartbeater.cc:503] Master 127.24.71.254:40165 was elected leader, sending a full tablet report...
I20260811 22:28:11.816991 7075 tablet_bootstrap.cc:492] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c: No bootstrap required, opened a new log
I20260811 22:28:11.817268 7075 ts_tablet_manager.cc:1397] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
I20260811 22:28:11.822443 7075 raft_consensus.cc:348] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } }
I20260811 22:28:11.822775 7075 raft_consensus.cc:374] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:11.822824 7075 raft_consensus.cc:729] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b23ffb96562649b3a1371daea3efe43c, State: Initialized, Role: FOLLOWER
I20260811 22:28:11.823446 7075 consensus_queue.cc:260] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } }
I20260811 22:28:11.824239 6746 heartbeater.cc:503] Master 127.24.71.254:40165 was elected leader, sending a full tablet report...
I20260811 22:28:11.824666 7075 ts_tablet_manager.cc:1428] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:28:11.859583 7080 raft_consensus.cc:482] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:11.859905 7080 raft_consensus.cc:504] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } }
I20260811 22:28:11.861393 7080 leader_election.cc:288] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b23ffb96562649b3a1371daea3efe43c (127.24.71.193:42569), eab8daf8346f488b93b435279054faad (127.24.71.194:42255)
I20260811 22:28:11.868072 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.862635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:34061 (local address 127.24.71.194:42255)
0811 22:28:11.862783 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:11.862787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:11.863113 (+ 326us) server_negotiation.cc:407] Connection header received
0811 22:28:11.863333 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:11.863337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:11.863415 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:11.863484 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:11.863966 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.864768 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.866400 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.867142 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.867257 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.867323 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:11.867802 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 22:28:11.867920 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:28:11.868827 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.862143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:42255 (local address 127.24.71.195:34061)
0811 22:28:11.863015 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:11.863033 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:11.863196 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:11.863637 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:11.863640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:11.863658 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:11.863780 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.863786 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.864915 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.864918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.866154 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.866165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.867512 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.867517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.867631 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.867659 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:28:11.868546 (+ 887us) client_negotiation.cc:240] Negotiation successful
0811 22:28:11.868649 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"thread_start_us":101,"threads_started":1}
I20260811 22:28:11.869266 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.861868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:49599 (local address 127.24.71.193:42569)
0811 22:28:11.862107 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:11.862112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:11.862445 (+ 333us) server_negotiation.cc:407] Connection header received
0811 22:28:11.862755 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:11.862759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:11.862827 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:11.862889 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:11.863491 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.864398 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.866259 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.867136 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.867257 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.867323 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:11.869020 (+ 1697us) server_negotiation.cc:299] Negotiation successful
0811 22:28:11.869123 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260811 22:28:11.869679 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.861671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:42569 (local address 127.24.71.195:49599)
0811 22:28:11.862303 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:11.862320 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:11.862581 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:11.863083 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:11.863089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:11.863114 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:11.863343 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.863352 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.864559 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.864563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.866063 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:11.866077 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.868719 (+ 2642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.868723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:11.868830 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.868861 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:28:11.869419 (+ 558us) client_negotiation.cc:240] Negotiation successful
0811 22:28:11.869527 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"spinlock_wait_cycles":37120,"thread_start_us":145,"threads_started":1}
I20260811 22:28:11.869766 6842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "559092d5fa7746b39766b19fe91a3a7a" candidate_uuid: "5deb6b047da5465182048a3aaf0fd053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eab8daf8346f488b93b435279054faad" is_pre_election: true
I20260811 22:28:11.870198 6842 raft_consensus.cc:2454] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5deb6b047da5465182048a3aaf0fd053 in term 0.
I20260811 22:28:11.870354 6682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "559092d5fa7746b39766b19fe91a3a7a" candidate_uuid: "5deb6b047da5465182048a3aaf0fd053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b23ffb96562649b3a1371daea3efe43c" is_pre_election: true
I20260811 22:28:11.870644 6682 raft_consensus.cc:2454] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5deb6b047da5465182048a3aaf0fd053 in term 0.
I20260811 22:28:11.870800 6924 leader_election.cc:302] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5deb6b047da5465182048a3aaf0fd053, eab8daf8346f488b93b435279054faad; no voters:
I20260811 22:28:11.871181 7080 raft_consensus.cc:2781] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:28:11.871271 7080 raft_consensus.cc:482] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:28:11.871322 7080 raft_consensus.cc:3037] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:11.872867 7080 raft_consensus.cc:504] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } }
I20260811 22:28:11.873320 7080 leader_election.cc:288] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [CANDIDATE]: Term 1 election: Requested vote from peers b23ffb96562649b3a1371daea3efe43c (127.24.71.193:42569), eab8daf8346f488b93b435279054faad (127.24.71.194:42255)
I20260811 22:28:11.873739 6682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "559092d5fa7746b39766b19fe91a3a7a" candidate_uuid: "5deb6b047da5465182048a3aaf0fd053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b23ffb96562649b3a1371daea3efe43c"
I20260811 22:28:11.873850 6842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "559092d5fa7746b39766b19fe91a3a7a" candidate_uuid: "5deb6b047da5465182048a3aaf0fd053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eab8daf8346f488b93b435279054faad"
I20260811 22:28:11.873873 6682 raft_consensus.cc:3037] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:11.874001 6842 raft_consensus.cc:3037] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:11.875340 6682 raft_consensus.cc:2454] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5deb6b047da5465182048a3aaf0fd053 in term 1.
I20260811 22:28:11.875402 6842 raft_consensus.cc:2454] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5deb6b047da5465182048a3aaf0fd053 in term 1.
I20260811 22:28:11.875777 6926 leader_election.cc:302] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5deb6b047da5465182048a3aaf0fd053, b23ffb96562649b3a1371daea3efe43c; no voters:
I20260811 22:28:11.876001 7080 raft_consensus.cc:2781] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:28:11.876282 7080 raft_consensus.cc:686] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 1 LEADER]: Becoming Leader. State: Replica: 5deb6b047da5465182048a3aaf0fd053, State: Running, Role: LEADER
I20260811 22:28:11.876721 7080 consensus_queue.cc:237] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } }
I20260811 22:28:11.880997 6531 catalog_manager.cc:5495] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5deb6b047da5465182048a3aaf0fd053 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "5deb6b047da5465182048a3aaf0fd053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5deb6b047da5465182048a3aaf0fd053" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 41373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 } health_report { overall_health: UNKNOWN } } }
W20260811 22:28:11.882879 6747 tablet.cc:2369] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:28:11.903338 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:11.898566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54764 (local address 127.24.71.195:41373)
0811 22:28:11.898689 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:11.898696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:11.898717 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:28:11.898889 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:11.898893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:11.898977 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:11.899128 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:28:11.899410 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.900258 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.901490 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:11.901819 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:11.901937 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:11.902294 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:28:11.902910 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:28:11.903046 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:11.903120 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 22:28:11.903188 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:28:11.911021 6842 raft_consensus.cc:1261] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [term 1 FOLLOWER]: Refusing update from remote peer 5deb6b047da5465182048a3aaf0fd053: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:11.911098 6682 raft_consensus.cc:1261] T 559092d5fa7746b39766b19fe91a3a7a P b23ffb96562649b3a1371daea3efe43c [term 1 FOLLOWER]: Refusing update from remote peer 5deb6b047da5465182048a3aaf0fd053: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:11.911893 7080 consensus_queue.cc:1035] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b23ffb96562649b3a1371daea3efe43c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 42569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:28:11.912192 7084 consensus_queue.cc:1035] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eab8daf8346f488b93b435279054faad" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 42255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:28:11.924225 7089 mvcc.cc:205] Tried to move back new op lower bound from 7317451947658702848 to 7317451947530960896. Current Snapshot: MvccSnapshot[applied={T|T < 7317451947658702848}]
I20260811 22:28:11.925680 7091 mvcc.cc:205] Tried to move back new op lower bound from 7317451947658702848 to 7317451947530960896. Current Snapshot: MvccSnapshot[applied={T|T < 7317451947658702848}]
I20260811 22:28:11.934235 7090 mvcc.cc:205] Tried to move back new op lower bound from 7317451947658702848 to 7317451947530960896. Current Snapshot: MvccSnapshot[applied={T|T < 7317451947658702848}]
I20260811 22:28:11.938740 6531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:28:11.938947 6531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:28:11.942332 6531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b0b8f3f46bf4a9e9a934032a769c18f: Sending DeleteTablet for 3 replicas of tablet 559092d5fa7746b39766b19fe91a3a7a
I20260811 22:28:11.943670 6818 tablet_service.cc:1514] Processing DeleteTablet for tablet 559092d5fa7746b39766b19fe91a3a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:11 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:41064
I20260811 22:28:11.944826 6976 tablet_service.cc:1514] Processing DeleteTablet for tablet 559092d5fa7746b39766b19fe91a3a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:11 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:54762
I20260811 22:28:11.945593 6665 tablet_service.cc:1514] Processing DeleteTablet for tablet 559092d5fa7746b39766b19fe91a3a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:11 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:55620
I20260811 22:28:11.947028 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 6571
I20260811 22:28:11.956096 7100 tablet_replica.cc:330] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: stopping tablet replica
I20260811 22:28:11.964119 7099 tablet_replica.cc:330] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: stopping tablet replica
I20260811 22:28:11.964372 7100 raft_consensus.cc:2229] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:28:11.964969 7100 raft_consensus.cc:2258] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:28:11.965389 7099 raft_consensus.cc:2229] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:28:11.965799 7099 raft_consensus.cc:2258] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:28:11.970078 7099 ts_tablet_manager.cc:1905] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:28:11.967787 7100 ts_tablet_manager.cc:1905] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:28:11.983147 7100 ts_tablet_manager.cc:1918] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:28:11.988852 7099 ts_tablet_manager.cc:1918] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:28:11.989173 7100 log.cc:1196] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/559092d5fa7746b39766b19fe91a3a7a
I20260811 22:28:11.989636 7100 ts_tablet_manager.cc:1939] T 559092d5fa7746b39766b19fe91a3a7a P 5deb6b047da5465182048a3aaf0fd053: Deleting consensus metadata
I20260811 22:28:11.991257 6517 catalog_manager.cc:4841] TS 5deb6b047da5465182048a3aaf0fd053 (127.24.71.195:41373): tablet 559092d5fa7746b39766b19fe91a3a7a (table test-table [id=ed18ecca73ac438daa839b21be86b6d0]) successfully deleted
I20260811 22:28:11.991621 7099 log.cc:1196] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/559092d5fa7746b39766b19fe91a3a7a
I20260811 22:28:11.992113 7099 ts_tablet_manager.cc:1939] T 559092d5fa7746b39766b19fe91a3a7a P eab8daf8346f488b93b435279054faad: Deleting consensus metadata
W20260811 22:28:11.992532 6517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:42569 (error 108)
I20260811 22:28:11.994812 6515 catalog_manager.cc:4841] TS eab8daf8346f488b93b435279054faad (127.24.71.194:42255): tablet 559092d5fa7746b39766b19fe91a3a7a (table test-table [id=ed18ecca73ac438daa839b21be86b6d0]) successfully deleted
I20260811 22:28:11.995203 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 6749
I20260811 22:28:12.004987 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:12.004499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:42569 (local address 127.0.0.1:55630)
0811 22:28:12.004766 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:12.004872 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
W20260811 22:28:12.005394 6517 catalog_manager.cc:4584] TS b23ffb96562649b3a1371daea3efe43c (127.24.71.193:42569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 559092d5fa7746b39766b19fe91a3a7a: Network error: Client connection negotiation failed: client connection to 127.24.71.193:42569: connect: Connection refused (error 111)
I20260811 22:28:12.010141 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 6897
I20260811 22:28:12.022846 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 6494
I20260811 22:28:12.034245 24863 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:28:12.380149 24863 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:28:12.380270 24863 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:28:12.726044 24863 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:28:12.726 UTC [5455] LOG: received smart shutdown request
2026-08-11 22:28:12.730 UTC [5455] LOG: background worker "logical replication launcher" (PID 5465) exited with exit code 1
2026-08-11 22:28:12.731 UTC [5459] LOG: shutting down
2026-08-11 22:28:12.915 UTC [5455] LOG: database system is shut down
2026-08-11T22:28:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51549 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:28:13 dist-test-slave-mvvj krb5kdc[7115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:28:13 dist-test-slave-mvvj krb5kdc[7115](info): set up 2 sockets
Aug 11 22:28:13 dist-test-slave-mvvj krb5kdc[7115](info): commencing operation
krb5kdc: starting...
W20260811 22:28:14.151157 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:28:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:28:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:28:14.288232 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:44193
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:44193,127.24.71.253:39339,127.24.71.252:32817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:14.405126 7129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:14.405448 7129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:14.405525 7129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:14.409596 7129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:14.409686 7129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:14.409724 7129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:14.409759 7129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:14.409795 7129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:14.415144 7129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:44193,127.24.71.253:39339,127.24.71.252:32817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:44193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:14.416705 7129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:14.417789 7129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:14.426887 7134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:14.426901 7135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:14.426887 7137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:14.427122 7129 server_base.cc:1034] running on GCE node
I20260811 22:28:14.427726 7129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:14.428897 7129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:14.430097 7129 hybrid_clock.cc:648] HybridClock initialized: now 1786487294430074 us; error 35 us; skew 500 ppm
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:14.433161 7129 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:28:14.434093 7129 webserver.cc:460] Webserver started at http://127.24.71.254:34089/ using document root <none> and password file <none>
I20260811 22:28:14.434680 7129 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:14.434744 7129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:14.435052 7129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:14.437359 7129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "6a2e1948d4184b41818c686802be45f8"
format_stamp: "Formatted at 2026-08-11 22:28:14 on dist-test-slave-mvvj"
server_key: "cdf43bfc491ad3cae09874c0db8a43d3"
server_key_iv: "6ad7e31beb20bfb9639b6131430f9180"
server_key_version: "encryptionkey@0"
I20260811 22:28:14.438004 7129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "6a2e1948d4184b41818c686802be45f8"
format_stamp: "Formatted at 2026-08-11 22:28:14 on dist-test-slave-mvvj"
server_key: "cdf43bfc491ad3cae09874c0db8a43d3"
server_key_iv: "6ad7e31beb20bfb9639b6131430f9180"
server_key_version: "encryptionkey@0"
I20260811 22:28:14.442184 7129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:28:14.444819 7144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:14.446120 7129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:28:14.446264 7129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "6a2e1948d4184b41818c686802be45f8"
format_stamp: "Formatted at 2026-08-11 22:28:14 on dist-test-slave-mvvj"
server_key: "cdf43bfc491ad3cae09874c0db8a43d3"
server_key_iv: "6ad7e31beb20bfb9639b6131430f9180"
server_key_version: "encryptionkey@0"
I20260811 22:28:14.446391 7129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:14.463552 7129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:14.466938 7129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:14.467160 7129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:14.475060 7129 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:44193
I20260811 22:28:14.475071 7200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:44193 every 8 connection(s)
I20260811 22:28:14.476186 7129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:28:14.479228 7201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:14.483227 7201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:28:14.485966 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 7129
I20260811 22:28:14.486105 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:28:14.486431 24863 external_mini_cluster.cc:1428] Setting key e7de11d66330f9e0cab25eeaf1a069f9
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.253@KRBTEST.COM" created.
I20260811 22:28:14.499706 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.485595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:44193 (local address 127.0.0.1:51470)
0811 22:28:14.486115 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.486169 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:14.487016 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:14.487867 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:14.487876 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:14.488225 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:14.488593 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.488616 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.489780 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.489786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.491033 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.491048 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.491790 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.491796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.491984 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.492663 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:14.492691 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:14.494449 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:14.496546 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:14.496554 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:14.496564 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:14.496794 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:14.497160 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:14.497165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:14.497167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:14.497285 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:14.497915 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:14.497921 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:14.498073 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 22:28:14.498268 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 22:28:14.498709 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":92,"thread_start_us":158,"threads_started":1}
I20260811 22:28:14.499706 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.485595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.24.71.254:44193)
0811 22:28:14.486388 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:14.486399 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:14.486446 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:28:14.487347 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:14.487356 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:14.487597 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:14.487796 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:14.488769 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.489646 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.491163 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.491581 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.491738 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.494637 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:14.494643 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:14.494653 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:14.494673 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:14.496397 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:14.496948 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:14.496953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:14.496954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:14.497062 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:14.497389 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:14.497393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:14.497396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:14.497787 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:14.497978 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:14.498346 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 22:28:14.498629 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"spinlock_wait_cycles":768,"thread_start_us":181,"threads_started":1}
I20260811 22:28:14.502161 7201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:28:14.503031 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.502671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48478)
0811 22:28:14.502833 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.502934 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:28:14.503264 7145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
I20260811 22:28:14.504244 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.503957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48484)
0811 22:28:14.504098 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.504160 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:28:14.504415 7201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:39339: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab.
Entry for principal kudu/127.24.71.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab.
I20260811 22:28:14.517421 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.253:39339
--webserver_interface=127.24.71.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab
--principal=kudu/127.24.71.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:44193,127.24.71.253:39339,127.24.71.252:32817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:28:14.553691 7201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } attempt: 1
I20260811 22:28:14.554702 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.554354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48500)
0811 22:28:14.554544 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.554617 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:28:14.555637 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.555320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48508)
0811 22:28:14.555510 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.555572 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260811 22:28:14.555764 7201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:39339: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
I20260811 22:28:14.623967 7201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } attempt: 2
I20260811 22:28:14.624927 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.624565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48510)
0811 22:28:14.624750 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.624828 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:28:14.625859 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.625617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48514)
0811 22:28:14.625736 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.625791 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:28:14.626063 7201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:39339: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
W20260811 22:28:14.634873 7208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:14.635135 7208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:14.635213 7208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:14.639369 7208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:14.639430 7208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:14.639451 7208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:14.639467 7208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:14.639487 7208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:14.645100 7208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:44193,127.24.71.253:39339,127.24.71.252:32817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab
--principal=kudu/127.24.71.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.253:39339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.71.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:14.646260 7208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:14.647329 7208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:28:14.656260 7208 server_base.cc:1034] running on GCE node
W20260811 22:28:14.656270 7216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:14.656279 7214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:14.656473 7213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:14.657023 7208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:14.658133 7208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:14.659297 7208 hybrid_clock.cc:648] HybridClock initialized: now 1786487294659261 us; error 50 us; skew 500 ppm
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:14.662240 7208 init.cc:377] Logged in from keytab as kudu/127.24.71.253@KRBTEST.COM (short username kudu)
I20260811 22:28:14.663359 7208 webserver.cc:460] Webserver started at http://127.24.71.253:36847/ using document root <none> and password file <none>
I20260811 22:28:14.663993 7208 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:14.664065 7208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:14.664299 7208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:14.666399 7208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/data/instance:
uuid: "eba86477664e48098048904ed3dd6384"
format_stamp: "Formatted at 2026-08-11 22:28:14 on dist-test-slave-mvvj"
server_key: "88069dc4c532f623f1f695d2b189379c"
server_key_iv: "b858798ac5bc455d1db0b69cb40cfcab"
server_key_version: "encryptionkey@0"
I20260811 22:28:14.666934 7208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/wal/instance:
uuid: "eba86477664e48098048904ed3dd6384"
format_stamp: "Formatted at 2026-08-11 22:28:14 on dist-test-slave-mvvj"
server_key: "88069dc4c532f623f1f695d2b189379c"
server_key_iv: "b858798ac5bc455d1db0b69cb40cfcab"
server_key_version: "encryptionkey@0"
I20260811 22:28:14.670742 7208 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:28:14.673352 7223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:14.674696 7208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:28:14.674840 7208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/wal
uuid: "eba86477664e48098048904ed3dd6384"
format_stamp: "Formatted at 2026-08-11 22:28:14 on dist-test-slave-mvvj"
server_key: "88069dc4c532f623f1f695d2b189379c"
server_key_iv: "b858798ac5bc455d1db0b69cb40cfcab"
server_key_version: "encryptionkey@0"
I20260811 22:28:14.674970 7208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:14.717304 7201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } attempt: 3
I20260811 22:28:14.718374 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.717994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48528)
0811 22:28:14.718180 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.718262 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:28:14.719254 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.719031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48540)
0811 22:28:14.719140 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.719195 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 22:28:14.719422 7201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:39339: Network error: Client connection negotiation failed: client connection to 127.24.71.253:39339: connect: Connection refused (error 111)
I20260811 22:28:14.722796 7208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:14.726590 7208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:14.726842 7208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:14.734983 7208 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.253:39339
I20260811 22:28:14.735009 7280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.253:39339 every 8 connection(s)
I20260811 22:28:14.736159 7208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
I20260811 22:28:14.739310 7281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:14.743366 7281 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:28:14.745724 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 7208
I20260811 22:28:14.745858 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-1/wal/instance
I20260811 22:28:14.746205 24863 external_mini_cluster.cc:1428] Setting key a22cb7eeef18dc09dbdcbff89ba31db6
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.252@KRBTEST.COM" created.
I20260811 22:28:14.759650 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.745410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.24.71.254:44193)
0811 22:28:14.745577 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:14.745581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:14.746635 (+ 1054us) server_negotiation.cc:407] Connection header received
0811 22:28:14.747646 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:14.747649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:14.747714 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:14.747826 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:14.749598 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.750188 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.752096 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.752386 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.752482 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.755637 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:14.755641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:14.755643 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:14.755669 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:14.757347 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:14.757911 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:14.757915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:14.757917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:14.758008 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:14.758387 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:14.758390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:14.758392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:14.758603 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:14.758715 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:14.759326 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 22:28:14.759470 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:28:14.760787 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.745839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:44193 (local address 127.0.0.1:51484)
0811 22:28:14.746471 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.746518 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:14.747427 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:14.748093 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:14.748115 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:14.748632 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:14.749377 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.749399 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.750336 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.750343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.751951 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.751967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.752595 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.752603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.752863 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.753657 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:14.753686 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:14.755439 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:14.757486 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:14.757493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:14.757511 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:14.757795 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:14.758115 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:14.758118 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:14.758120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:14.758274 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:14.758730 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:14.758738 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:14.759020 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 22:28:14.759258 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 22:28:14.759634 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":174,"threads_started":1}
I20260811 22:28:14.762300 7281 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
I20260811 22:28:14.771569 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.762850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48554)
0811 22:28:14.762995 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.763010 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:14.763093 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:14.763758 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:14.763761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:14.763976 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:14.764055 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.764061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.765033 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.765036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.765553 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.765561 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.766164 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.766168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.766264 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.766796 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:14.766807 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:14.768050 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:14.770000 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:14.770003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:14.770005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:14.770193 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:14.770489 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:14.770493 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:14.770495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:14.770571 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:14.771035 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:14.771038 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:14.771122 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 22:28:14.771233 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:28:14.771366 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:28:14.771605 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.763022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.24.71.253:39339)
0811 22:28:14.763416 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:14.763427 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:14.763447 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:28:14.763483 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:14.763488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:14.763636 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:14.763781 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:14.764165 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.764898 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.765667 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.766039 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.766156 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.768185 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:14.768194 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:14.768206 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:14.768226 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:14.769822 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:14.770303 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:14.770309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:14.770311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:14.770392 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:14.770660 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:14.770665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:14.770667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:14.770926 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:14.771075 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:14.771334 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 22:28:14.771464 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":200,"threads_started":1}
I20260811 22:28:14.773984 7281 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:28:14.774749 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.774463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40120)
0811 22:28:14.774591 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.774675 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:28:14.775040 7227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
I20260811 22:28:14.776202 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.775980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40128)
0811 22:28:14.776089 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.776141 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:28:14.776403 7281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:32817: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab.
Entry for principal kudu/127.24.71.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab.
I20260811 22:28:14.779066 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.252:32817
--webserver_interface=127.24.71.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab
--principal=kudu/127.24.71.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:44193,127.24.71.253:39339,127.24.71.252:32817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:28:14.825652 7281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } attempt: 1
I20260811 22:28:14.826610 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.826252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40130)
0811 22:28:14.826432 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.826510 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:28:14.827417 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.827181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40144)
0811 22:28:14.827293 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.827347 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:28:14.827608 7281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:32817: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
I20260811 22:28:14.862690 7201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } attempt: 4
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
I20260811 22:28:14.873737 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.863480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48566)
0811 22:28:14.863762 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.863779 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:14.863988 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:14.864308 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:14.864311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:14.864633 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:14.864781 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.864789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.865992 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.865998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.867052 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.867065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.867604 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.867608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.867720 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.868310 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:14.868325 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:14.869956 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:14.872188 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:14.872191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:14.872192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:14.872408 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:14.872740 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:14.872742 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:14.872744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:14.872823 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:14.873293 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:14.873295 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:14.873387 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 22:28:14.873470 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:28:14.873597 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260811 22:28:14.873888 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.863633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48566 (local address 127.24.71.253:39339)
0811 22:28:14.863776 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:14.863782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:14.863901 (+ 119us) server_negotiation.cc:407] Connection header received
0811 22:28:14.864126 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:14.864134 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:14.864186 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:14.864284 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:14.864955 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.865853 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.867201 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.867462 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.867583 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.870131 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:14.870138 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:14.870144 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:14.870183 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:14.872064 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:14.872519 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:14.872525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:14.872527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:14.872633 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:14.872920 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:14.872924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:14.872925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:14.873190 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:14.873307 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:14.873563 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 22:28:14.873710 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:28:14.874799 7201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:28:14.875545 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.875303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40150)
0811 22:28:14.875423 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.875480 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 22:28:14.876262 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.876058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40160)
0811 22:28:14.876162 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.876207 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 22:28:14.876415 7201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:32817: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
I20260811 22:28:14.895793 7281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } attempt: 2
I20260811 22:28:14.896797 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.896417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40170)
0811 22:28:14.896616 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.896695 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 22:28:14.897675 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.897400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40174)
0811 22:28:14.897535 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.897597 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:28:14.897866 7281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:32817: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
W20260811 22:28:14.904426 7289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:14.904695 7289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:14.904790 7289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:14.908929 7289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:14.908993 7289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:14.909013 7289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:14.909030 7289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:14.909082 7289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:14.914563 7289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:44193,127.24.71.253:39339,127.24.71.252:32817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab
--principal=kudu/127.24.71.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.252:32817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.71.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:14.915700 7289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:14.916779 7289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:14.926116 7297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:14.926115 7294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:14.926123 7295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:14.926538 7289 server_base.cc:1034] running on GCE node
I20260811 22:28:14.927138 7289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:14.928324 7289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:14.929520 7289 hybrid_clock.cc:648] HybridClock initialized: now 1786487294929503 us; error 29 us; skew 500 ppm
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:14.932482 7289 init.cc:377] Logged in from keytab as kudu/127.24.71.252@KRBTEST.COM (short username kudu)
I20260811 22:28:14.933441 7289 webserver.cc:460] Webserver started at http://127.24.71.252:38687/ using document root <none> and password file <none>
I20260811 22:28:14.934046 7289 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:14.934144 7289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:14.934391 7289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:14.935576 7201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } attempt: 1
I20260811 22:28:14.936446 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.936107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40178)
0811 22:28:14.936268 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.936325 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:28:14.936913 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/data/instance:
uuid: "f51ac26e2c474f8093653ff3c34af394"
format_stamp: "Formatted at 2026-08-11 22:28:14 on dist-test-slave-mvvj"
server_key: "afc55e6f0de95eb85e06812ba1e12165"
server_key_iv: "acb0871877a21b694676e1e6ca3800d8"
server_key_version: "encryptionkey@0"
I20260811 22:28:14.937206 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.936982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40192)
0811 22:28:14.937099 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.937151 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:28:14.937386 7201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:32817: Network error: Client connection negotiation failed: client connection to 127.24.71.252:32817: connect: Connection refused (error 111)
I20260811 22:28:14.937606 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/wal/instance:
uuid: "f51ac26e2c474f8093653ff3c34af394"
format_stamp: "Formatted at 2026-08-11 22:28:14 on dist-test-slave-mvvj"
server_key: "afc55e6f0de95eb85e06812ba1e12165"
server_key_iv: "acb0871877a21b694676e1e6ca3800d8"
server_key_version: "encryptionkey@0"
I20260811 22:28:14.941758 7289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:28:14.944465 7304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:14.945703 7289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:14.945850 7289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/wal
uuid: "f51ac26e2c474f8093653ff3c34af394"
format_stamp: "Formatted at 2026-08-11 22:28:14 on dist-test-slave-mvvj"
server_key: "afc55e6f0de95eb85e06812ba1e12165"
server_key_iv: "acb0871877a21b694676e1e6ca3800d8"
server_key_version: "encryptionkey@0"
I20260811 22:28:14.945974 7289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:14.969967 7289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:14.973412 7289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:14.973634 7289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:14.982240 7289 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.252:32817
I20260811 22:28:14.982249 7358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.252:32817 every 8 connection(s)
I20260811 22:28:14.983371 7289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
I20260811 22:28:14.986399 7360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:14.986994 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 7289
I20260811 22:28:14.987123 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786487121090521-24863-0/minicluster-data/master-2/wal/instance
I20260811 22:28:14.987459 24863 external_mini_cluster.cc:1428] Setting key 85ef744527c37492742cab018bcb0b4f
I20260811 22:28:14.989075 7281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } attempt: 3
I20260811 22:28:14.991485 7360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
Aug 11 22:28:14 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:15.002977 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.989624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40212)
0811 22:28:14.989909 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.989925 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:14.990022 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:14.991263 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:14.991266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:14.991482 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:14.991615 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.991624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.993306 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.993310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.994142 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.994154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.995394 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.995398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.995503 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.996131 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:14.996154 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:14.997782 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:15.000305 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.000309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.000311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.000583 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.001512 (+ 929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.001517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.001520 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.001628 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.002395 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:15.002401 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:15.002533 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 22:28:15.002646 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:28:15.002804 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260811 22:28:15.004420 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.993664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51500 (local address 127.24.71.254:44193)
0811 22:28:14.993813 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:14.993818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:14.994221 (+ 403us) server_negotiation.cc:407] Connection header received
0811 22:28:14.994429 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:14.994432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:14.994487 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:14.994584 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:14.995296 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.996086 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.997076 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.997450 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.997567 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.000582 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:15.000586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:15.000590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:15.000623 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.002453 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.002888 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.002892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.002894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.002992 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.003369 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.003373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.003375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.003597 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.003714 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.004131 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.004263 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:28:15.004555 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.989911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.24.71.252:32817)
0811 22:28:14.990220 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:14.990229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:14.990244 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:28:14.990547 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:14.990560 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:14.990952 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:14.991737 (+ 785us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:14.991921 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.993127 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.994293 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.995236 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.995433 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.997939 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:14.997947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:14.997960 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:14.998008 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.000179 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.001203 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.001213 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.001221 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.001376 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.001741 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.001746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.001748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.002280 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.002414 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.002822 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.003334 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":199,"server-negotiator.queue_time_us":221,"spinlock_wait_cycles":6016,"thread_start_us":79,"threads_started":1}
I20260811 22:28:15.004554 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.993616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:44193 (local address 127.0.0.1:51500)
0811 22:28:14.994094 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:14.994145 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:14.994314 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:14.994638 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:14.994645 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:14.995030 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:14.995149 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.995155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.996232 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.996235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.996925 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:14.996937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.997724 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.997727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:14.997804 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.998316 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:14.998335 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:15.000405 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:15.002552 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.002556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.002561 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.002766 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.003151 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.003155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.003158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.003248 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.003728 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:15.003737 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:15.003908 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 22:28:15.004069 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 22:28:15.004263 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":149,"threads_started":1}
I20260811 22:28:15.004544 7201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } attempt: 2
I20260811 22:28:15.005088 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:14.989026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.24.71.252:32817)
0811 22:28:14.989638 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:14.989658 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:14.989705 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:28:14.990535 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:14.990556 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:14.990948 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:14.991301 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:14.992219 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.993060 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.994293 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:14.995236 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:14.995426 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:14.998965 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:14.998970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:14.998975 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:14.999008 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.001648 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.002193 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.002196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.002198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.002274 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.003428 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.003431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.003433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.003650 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.003807 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.004808 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.004927 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":595,"server-negotiator.queue_time_us":421,"spinlock_wait_cycles":21888,"thread_start_us":133,"threads_started":1}
I20260811 22:28:15.006620 7360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:28:15.009958 7281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384: Bootstrap starting.
Aug 11 22:28:15 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
Aug 11 22:28:15 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
I20260811 22:28:15.013182 7281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:15.014219 7281 log.cc:824] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:28:15 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:15.017202 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.007261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.24.71.253:39339)
0811 22:28:15.007394 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:15.007398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:15.007564 (+ 166us) server_negotiation.cc:407] Connection header received
0811 22:28:15.007746 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:15.007749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:15.007798 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:15.007883 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:15.008505 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.009264 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.010329 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.010670 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.010748 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.013055 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:15.013059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:15.013063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:15.013095 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.014955 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.015571 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.015574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.015576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.015665 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.016054 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.016057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.016059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.016370 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.016456 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.016872 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.017009 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:28:15.017263 7281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384: No bootstrap required, opened a new log
I20260811 22:28:15.018000 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.005174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40218)
0811 22:28:15.005362 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:15.005377 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:15.005451 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:15.006411 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:15.006414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:15.006621 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:15.006737 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:15.006745 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.008186 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.008190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:15.008990 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:15.009002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.009598 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.009615 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:15.009698 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.010332 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:15.010348 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:15.013115 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:15.016035 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.016038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.016040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.016597 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.016902 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.016905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.016907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.017014 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.017476 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:15.017480 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:15.017595 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 22:28:15.017674 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:28:15.017819 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":103}
I20260811 22:28:15.018009 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.007186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:39339 (local address 127.0.0.1:48578)
0811 22:28:15.007482 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:15.007497 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:15.007632 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:15.008038 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:15.008041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:15.008235 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:15.008361 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:15.008371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.009410 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.009413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:15.010182 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:15.010192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.010916 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.010922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:15.011024 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.011468 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:15.011482 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:15.012885 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:15.015085 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.015092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.015095 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.015409 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.015785 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.015791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.015796 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.015935 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.016557 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:15.016559 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:15.016673 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 22:28:15.017718 (+ 1045us) client_negotiation.cc:240] Negotiation successful
0811 22:28:15.017859 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230}
I20260811 22:28:15.018870 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.006096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.24.71.252:32817)
0811 22:28:15.006208 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:15.006212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:15.006226 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:28:15.006259 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:15.006262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:15.006307 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:15.006385 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:15.007305 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.008043 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.009148 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.009461 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.009765 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.013287 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:15.013291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:15.013294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:15.013320 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.015889 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.016713 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.016716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.016718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.016799 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.017139 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.017141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.017143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.017346 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.018545 (+ 1199us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.018617 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.018735 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:28:15.019109 7360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:28:15.019218 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.007568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.24.71.254:44193)
0811 22:28:15.007676 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:15.007681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:15.007877 (+ 196us) server_negotiation.cc:407] Connection header received
0811 22:28:15.008053 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:15.008056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:15.008102 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:15.008152 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:15.009033 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.009753 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.011810 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.012168 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.012229 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.015126 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:15.015130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:15.015135 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:15.015161 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.017007 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.017524 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.017528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.017529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.017621 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.017958 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.017962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.017963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.018344 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.018553 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.018942 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.019064 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":41}
I20260811 22:28:15.022521 7281 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } }
I20260811 22:28:15.022725 7201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8: Bootstrap starting.
I20260811 22:28:15.022789 7281 raft_consensus.cc:374] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:15.022850 7281 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba86477664e48098048904ed3dd6384, State: Initialized, Role: FOLLOWER
I20260811 22:28:15.023381 7281 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } }
I20260811 22:28:15.023952 7368 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.024164 7368 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:28:15.025662 7201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:15.026521 7201 log.cc:824] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:28:15 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
I20260811 22:28:15.027128 7281 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:28:15.029160 7201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8: No bootstrap required, opened a new log
Aug 11 22:28:15 dist-test-slave-mvvj krb5kdc[7115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
I20260811 22:28:15.031469 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.021064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.24.71.253:39339)
0811 22:28:15.021174 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:15.021179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:15.021193 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:28:15.021228 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:15.021231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:15.021272 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:15.021355 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:15.022135 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.022846 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.023978 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.024281 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.024358 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.027684 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:15.027689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:15.027691 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:15.027719 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.029452 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.030002 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.030005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.030007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.030099 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.030427 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.030430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.030432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.030638 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.030785 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.031178 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.031312 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:28:15.033800 7201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } }
I20260811 22:28:15.034145 7201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:15.034204 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.019881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.24.71.252:32817)
0811 22:28:15.025264 (+ 5383us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:15.025272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:15.025291 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:28:15.025338 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:15.025342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:15.025395 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:15.025485 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:15.026105 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.026778 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.027759 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.028120 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.028207 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.030833 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:15.030837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:15.030839 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:15.030862 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.032555 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.033045 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.033049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.033051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.033122 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.033361 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.033363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.033367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.033565 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.033638 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.033925 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.034042 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5266}
I20260811 22:28:15.034238 7201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a2e1948d4184b41818c686802be45f8, State: Initialized, Role: FOLLOWER
I20260811 22:28:15.034718 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.019772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:32817 (local address 127.0.0.1:40226)
0811 22:28:15.020048 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:15.020064 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:15.020142 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:15.025619 (+ 5477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:15.025624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:15.025846 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:15.025963 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:15.025984 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.026885 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.026889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:15.027631 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:15.027644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.028377 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.028381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:15.028492 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.029005 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:15.029021 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:15.030690 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:15.032698 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.032701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.032703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.032944 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.033209 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:15.033213 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:15.033215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:15.033283 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:15.033691 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:15.033694 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:15.033771 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 22:28:15.034388 (+ 617us) client_negotiation.cc:240] Negotiation successful
0811 22:28:15.034534 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260811 22:28:15.035059 7201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } }
I20260811 22:28:15.036077 7386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.036240 7386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:28:15.036583 7201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:28:15.037686 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:28:15.032809 7208 master_runner.cc:186] Error getting master registration for 127.24.71.254:44193: OK, instance_id { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" instance_seqno: 1786487294472804 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:28:15.037901 7208 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:28:15.038172 7360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394: Bootstrap starting.
I20260811 22:28:15.041033 7360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394: Neither blocks nor log segments found. Creating new log.
W20260811 22:28:15.042032 7398 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 22:28:15.042037 7360 log.cc:824] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394: Log is configured to *not* fsync() on all Append() calls
W20260811 22:28:15.042217 7398 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:28:15.044806 7360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394: No bootstrap required, opened a new log
W20260811 22:28:15.046480 7385 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 22:28:15.045068 7129 master_runner.cc:186] Error getting master registration for 127.24.71.252:32817: OK, instance_id { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" instance_seqno: 1786487294979666 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:28:15.048583 7129 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:28:15.049361 7360 raft_consensus.cc:348] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } }
I20260811 22:28:15.049716 7360 raft_consensus.cc:374] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:15.049844 7360 raft_consensus.cc:729] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51ac26e2c474f8093653ff3c34af394, State: Initialized, Role: FOLLOWER
I20260811 22:28:15.050563 7360 consensus_queue.cc:260] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } }
I20260811 22:28:15.052166 7360 sys_catalog.cc:564] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:28:15.052829 7385 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 22:28:15.058797 7423 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:28:15.059361 7423 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:28:15.059211 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.059614 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:28:15.063149 7289 master_runner.cc:263] No leader master found from master f51ac26e2c474f8093653ff3c34af394
I20260811 22:28:15.063227 7289 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:28:15.064419 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.041101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.24.71.253:39339)
0811 22:28:15.041218 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:15.041222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:15.046866 (+ 5644us) server_negotiation.cc:407] Connection header received
0811 22:28:15.047078 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:15.047081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:15.047131 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:15.047184 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:15.049697 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.050499 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.054059 (+ 3560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.054496 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.054595 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.057320 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:15.057325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:15.057327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:15.057368 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.059073 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.060308 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.060313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.060315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.060426 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.061254 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.061260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.061263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.061522 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.061641 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.064070 (+ 2429us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.064212 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:28:15.067353 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.042027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51524 (local address 127.24.71.254:44193)
0811 22:28:15.042168 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:15.042173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:15.048447 (+ 6274us) server_negotiation.cc:407] Connection header received
0811 22:28:15.048619 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:15.048623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:15.048675 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:15.048725 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:15.051713 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.052455 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.055220 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.055573 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.055654 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.060872 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:15.060879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:15.060881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:15.060912 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.062917 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.063549 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.063554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.063556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.063650 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.066148 (+ 2498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.066153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.066155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.066423 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.066557 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.067020 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.067156 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:28:15.070118 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:15.042856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40242 (local address 127.24.71.252:32817)
0811 22:28:15.042980 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:15.042985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:15.048935 (+ 5950us) server_negotiation.cc:407] Connection header received
0811 22:28:15.049181 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:15.049185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:15.049238 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:15.049294 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:15.050418 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.051187 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.063962 (+ 12775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:15.064341 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:15.064431 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:15.066201 (+ 1770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:15.066205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:15.066219 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:15.066248 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:15.067942 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.068453 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.068457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.068459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.068558 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:15.068847 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:15.068850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:15.068852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:15.069066 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:15.069206 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:15.069844 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 22:28:15.069958 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:28:15.075453 7411 raft_consensus.cc:482] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:15.075698 7411 raft_consensus.cc:504] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } }
I20260811 22:28:15.076851 7411 leader_election.cc:288] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a2e1948d4184b41818c686802be45f8 (127.24.71.254:44193), eba86477664e48098048904ed3dd6384 (127.24.71.253:39339)
I20260811 22:28:15.077503 7172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f51ac26e2c474f8093653ff3c34af394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a2e1948d4184b41818c686802be45f8" is_pre_election: true
I20260811 22:28:15.077687 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f51ac26e2c474f8093653ff3c34af394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba86477664e48098048904ed3dd6384" is_pre_election: true
I20260811 22:28:15.077857 7172 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51ac26e2c474f8093653ff3c34af394 in term 0.
I20260811 22:28:15.078006 7252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51ac26e2c474f8093653ff3c34af394 in term 0.
I20260811 22:28:15.078336 7308 leader_election.cc:302] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a2e1948d4184b41818c686802be45f8, f51ac26e2c474f8093653ff3c34af394; no voters:
I20260811 22:28:15.078613 7411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:28:15.078728 7411 raft_consensus.cc:482] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:28:15.078774 7411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:15.080181 7411 raft_consensus.cc:504] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } }
I20260811 22:28:15.080811 7172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f51ac26e2c474f8093653ff3c34af394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a2e1948d4184b41818c686802be45f8"
I20260811 22:28:15.080937 7172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:15.080987 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f51ac26e2c474f8093653ff3c34af394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba86477664e48098048904ed3dd6384"
I20260811 22:28:15.081099 7252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:15.082433 7252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51ac26e2c474f8093653ff3c34af394 in term 1.
I20260811 22:28:15.082444 7172 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51ac26e2c474f8093653ff3c34af394 in term 1.
I20260811 22:28:15.083200 7305 leader_election.cc:302] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eba86477664e48098048904ed3dd6384, f51ac26e2c474f8093653ff3c34af394; no voters:
I20260811 22:28:15.083571 7411 leader_election.cc:288] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [CANDIDATE]: Term 1 election: Requested vote from peers 6a2e1948d4184b41818c686802be45f8 (127.24.71.254:44193), eba86477664e48098048904ed3dd6384 (127.24.71.253:39339)
I20260811 22:28:15.083771 7411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:28:15.083935 7411 raft_consensus.cc:686] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [term 1 LEADER]: Becoming Leader. State: Replica: f51ac26e2c474f8093653ff3c34af394, State: Running, Role: LEADER
I20260811 22:28:15.084333 7411 consensus_queue.cc:237] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } }
I20260811 22:28:15.086266 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f51ac26e2c474f8093653ff3c34af394. Latest consensus state: current_term: 1 leader_uuid: "f51ac26e2c474f8093653ff3c34af394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.086445 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:15.086875 7428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:28:15.087888 7428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:28:15.095813 7252 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [term 1 FOLLOWER]: Refusing update from remote peer f51ac26e2c474f8093653ff3c34af394: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:15.096696 7411 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:28:15.101975 7368 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f51ac26e2c474f8093653ff3c34af394. Latest consensus state: current_term: 1 leader_uuid: "f51ac26e2c474f8093653ff3c34af394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.102113 7368 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:28:15.103977 7172 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [term 1 FOLLOWER]: Refusing update from remote peer f51ac26e2c474f8093653ff3c34af394: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:15.104625 7411 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:28:15.108686 7386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f51ac26e2c474f8093653ff3c34af394. Latest consensus state: current_term: 1 leader_uuid: "f51ac26e2c474f8093653ff3c34af394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.108811 7386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:28:15.120110 7426 sys_catalog.cc:455] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f51ac26e2c474f8093653ff3c34af394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.121918 7386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f51ac26e2c474f8093653ff3c34af394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.122058 7386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:28:15.122354 7426 sys_catalog.cc:458] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:15.122737 7430 mvcc.cc:205] Tried to move back new op lower bound from 7317451960704618496 to 7317451960669069312. Current Snapshot: MvccSnapshot[applied={T|T < 7317451960704618496}]
I20260811 22:28:15.124464 7433 mvcc.cc:205] Tried to move back new op lower bound from 7317451960704618496 to 7317451960669069312. Current Snapshot: MvccSnapshot[applied={T|T < 7317451960704618496}]
I20260811 22:28:15.125854 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f51ac26e2c474f8093653ff3c34af394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.125973 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:15.127007 7368 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f51ac26e2c474f8093653ff3c34af394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a2e1948d4184b41818c686802be45f8" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 44193 } } peers { permanent_uuid: "eba86477664e48098048904ed3dd6384" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 39339 } } peers { permanent_uuid: "f51ac26e2c474f8093653ff3c34af394" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 32817 } } }
I20260811 22:28:15.127125 7368 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:28:15.128037 7431 mvcc.cc:205] Tried to move back new op lower bound from 7317451960704618496 to 7317451960669069312. Current Snapshot: MvccSnapshot[applied={T|T < 7317451960704618496}]
I20260811 22:28:15.128336 7428 catalog_manager.cc:1347] Generated new cluster ID: 6c9962429e274fccb40e9d7e86ffb050
I20260811 22:28:15.128398 7428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:28:15.154103 7428 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:28:15.155646 7428 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:28:15.175423 7428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f51ac26e2c474f8093653ff3c34af394: Generated new TSK 0
I20260811 22:28:15.176535 7428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:28:16.043708 7398 catalog_manager.cc:1259] Loaded cluster ID: 6c9962429e274fccb40e9d7e86ffb050
I20260811 22:28:16.043802 7398 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8: loading cluster ID for follower catalog manager: success
I20260811 22:28:16.045620 7398 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8: acquiring CA information for follower catalog manager: success
I20260811 22:28:16.046888 7398 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6a2e1948d4184b41818c686802be45f8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:28:16.055374 7385 catalog_manager.cc:1259] Loaded cluster ID: 6c9962429e274fccb40e9d7e86ffb050
I20260811 22:28:16.055506 7385 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384: loading cluster ID for follower catalog manager: success
I20260811 22:28:16.057236 7385 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384: acquiring CA information for follower catalog manager: success
I20260811 22:28:16.058512 7385 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eba86477664e48098048904ed3dd6384: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:28:16.251157 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 7129
I20260811 22:28:16.261377 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 7208
I20260811 22:28:16.270536 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 7289
2026-08-11T22:28:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3234 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:28:16 dist-test-slave-mvvj krb5kdc[7446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:28:16 dist-test-slave-mvvj krb5kdc[7446](info): set up 2 sockets
Aug 11 22:28:16 dist-test-slave-mvvj krb5kdc[7446](info): commencing operation
krb5kdc: starting...
W20260811 22:28:17.401124 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:28:17 dist-test-slave-mvvj krb5kdc[7446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:28:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:28:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:28:17.552022 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:34635
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:39247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:34635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:17.673518 7459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:17.673835 7459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:17.673880 7459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:17.678051 7459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:17.678133 7459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:17.678153 7459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:17.678169 7459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:17.678184 7459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:17.683943 7459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:39247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:34635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:34635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:17.685096 7459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:17.686131 7459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:17.695097 7468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:17.695135 7465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:17.695147 7466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:17.695489 7459 server_base.cc:1034] running on GCE node
I20260811 22:28:17.696163 7459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:17.697307 7459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:17.698485 7459 hybrid_clock.cc:648] HybridClock initialized: now 1786487297698464 us; error 38 us; skew 500 ppm
Aug 11 22:28:17 dist-test-slave-mvvj krb5kdc[7446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487297, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:17.701586 7459 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:28:17.702621 7459 webserver.cc:460] Webserver started at http://127.24.71.254:40577/ using document root <none> and password file <none>
I20260811 22:28:17.703253 7459 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:17.703351 7459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:17.703601 7459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:17.705893 7459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "c17b689400fd4abea8838975a2bab9d6"
format_stamp: "Formatted at 2026-08-11 22:28:17 on dist-test-slave-mvvj"
server_key: "c892e0ec4c2825b8015928bc53157472"
server_key_iv: "8dab32e041cd4ff0216d71b54831bd5a"
server_key_version: "encryptionkey@0"
I20260811 22:28:17.706539 7459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "c17b689400fd4abea8838975a2bab9d6"
format_stamp: "Formatted at 2026-08-11 22:28:17 on dist-test-slave-mvvj"
server_key: "c892e0ec4c2825b8015928bc53157472"
server_key_iv: "8dab32e041cd4ff0216d71b54831bd5a"
server_key_version: "encryptionkey@0"
I20260811 22:28:17.710795 7459 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:28:17.713718 7475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:17.715167 7459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:28:17.715350 7459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "c17b689400fd4abea8838975a2bab9d6"
format_stamp: "Formatted at 2026-08-11 22:28:17 on dist-test-slave-mvvj"
server_key: "c892e0ec4c2825b8015928bc53157472"
server_key_iv: "8dab32e041cd4ff0216d71b54831bd5a"
server_key_version: "encryptionkey@0"
I20260811 22:28:17.715505 7459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:17.736651 7459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:17.740536 7459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:17.740787 7459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:17.749290 7459 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:34635
I20260811 22:28:17.749311 7530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:34635 every 8 connection(s)
I20260811 22:28:17.750438 7459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:28:17.750756 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 7459
I20260811 22:28:17.750886 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:28:17.751478 24863 external_mini_cluster.cc:1428] Setting key e2b8cac666020f922b730296793f5e58
I20260811 22:28:17.754225 7531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:17.760545 7531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6: Bootstrap starting.
Aug 11 22:28:17 dist-test-slave-mvvj krb5kdc[7446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:17.763058 7531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:17.763952 7531 log.cc:824] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:17.766270 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:17.753080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45204 (local address 127.24.71.254:34635)
0811 22:28:17.753591 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:17.753606 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:17.753645 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:28:17.754367 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:17.754396 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:17.754740 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:17.755103 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:17.756013 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:17.757060 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:17.757932 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:17.758435 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:17.758629 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:17.761227 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:17.761232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:17.761246 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:17.761276 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:17.763061 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:17.763585 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:17.763590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:17.763595 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:17.763708 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:17.764046 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:17.764049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:17.764053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:17.764590 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:17.764796 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:17.765171 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 22:28:17.765506 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":171,"threads_started":1}
I20260811 22:28:17.766448 7531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6: No bootstrap required, opened a new log
I20260811 22:28:17.769384 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:28:17.770987 7531 raft_consensus.cc:348] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17b689400fd4abea8838975a2bab9d6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34635 } }
I20260811 22:28:17.771286 7531 raft_consensus.cc:374] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:17.771358 7531 raft_consensus.cc:729] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17b689400fd4abea8838975a2bab9d6, State: Initialized, Role: FOLLOWER
I20260811 22:28:17.771811 7531 consensus_queue.cc:260] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17b689400fd4abea8838975a2bab9d6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34635 } }
I20260811 22:28:17.772105 7531 raft_consensus.cc:388] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:28:17.772173 7531 raft_consensus.cc:482] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:28:17.772267 7531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:17.773380 7531 raft_consensus.cc:504] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17b689400fd4abea8838975a2bab9d6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34635 } }
I20260811 22:28:17.773712 7531 leader_election.cc:302] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c17b689400fd4abea8838975a2bab9d6; no voters:
I20260811 22:28:17.774017 7531 leader_election.cc:288] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:28:17.774181 7536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:28:17.774436 7536 raft_consensus.cc:686] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [term 1 LEADER]: Becoming Leader. State: Replica: c17b689400fd4abea8838975a2bab9d6, State: Running, Role: LEADER
I20260811 22:28:17.774782 7536 consensus_queue.cc:237] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17b689400fd4abea8838975a2bab9d6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34635 } }
I20260811 22:28:17.775467 7531 sys_catalog.cc:564] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:28:17.776721 7538 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c17b689400fd4abea8838975a2bab9d6. Latest consensus state: current_term: 1 leader_uuid: "c17b689400fd4abea8838975a2bab9d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17b689400fd4abea8838975a2bab9d6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34635 } } }
I20260811 22:28:17.776726 7537 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c17b689400fd4abea8838975a2bab9d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17b689400fd4abea8838975a2bab9d6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34635 } } }
I20260811 22:28:17.776942 7537 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:17.777251 7538 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:17.777868 7545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:28:17.780938 7545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:28:17.786971 7545 catalog_manager.cc:1347] Generated new cluster ID: f63ea9c0c7e54fb4b6873075f5c317b8
I20260811 22:28:17.787056 7545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:28:17.841359 7545 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:28:17.842448 7545 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:28:17.853530 7545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c17b689400fd4abea8838975a2bab9d6: Generated new TSK 0
I20260811 22:28:17.854324 7545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:28:17.877671 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 7459
2026-08-11T22:28:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1606 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:28:17 dist-test-slave-mvvj krb5kdc[7561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:28:17 dist-test-slave-mvvj krb5kdc[7561](info): set up 2 sockets
Aug 11 22:28:17 dist-test-slave-mvvj krb5kdc[7561](info): commencing operation
krb5kdc: starting...
W20260811 22:28:19.000965 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:28:19 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:28:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:28:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:28:19.144959 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42435
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:19.264925 7577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:19.265272 7577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:19.265345 7577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:19.269459 7577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:19.269565 7577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:19.269605 7577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:19.269666 7577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:19.269706 7577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:19.275192 7577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:19.276434 7577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:19.277534 7577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:28:19.286796 7577 server_base.cc:1034] running on GCE node
W20260811 22:28:19.286809 7585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:19.286839 7588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:19.286809 7583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:19.287670 7577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:19.288944 7577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:19.290158 7577 hybrid_clock.cc:648] HybridClock initialized: now 1786487299290131 us; error 41 us; skew 500 ppm
Aug 11 22:28:19 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487299, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:19.298137 7577 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:28:19.299279 7577 webserver.cc:460] Webserver started at http://127.24.71.254:33199/ using document root <none> and password file <none>
I20260811 22:28:19.299974 7577 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:19.300076 7577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:19.300303 7577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:19.302649 7577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "8a812062dbb4422c82d550f502772fb3"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "a3992af0def569d2a82f9783d70c5db2"
server_key_iv: "6de755bd13a36ce46295730dcbd244a3"
server_key_version: "encryptionkey@0"
I20260811 22:28:19.303336 7577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "8a812062dbb4422c82d550f502772fb3"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "a3992af0def569d2a82f9783d70c5db2"
server_key_iv: "6de755bd13a36ce46295730dcbd244a3"
server_key_version: "encryptionkey@0"
I20260811 22:28:19.307654 7577 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:28:19.310653 7595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:19.312145 7577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:19.312301 7577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "8a812062dbb4422c82d550f502772fb3"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "a3992af0def569d2a82f9783d70c5db2"
server_key_iv: "6de755bd13a36ce46295730dcbd244a3"
server_key_version: "encryptionkey@0"
I20260811 22:28:19.312439 7577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:19.326473 7577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:19.330287 7577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:19.330518 7577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:19.338183 7577 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:42435
I20260811 22:28:19.338647 7651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:42435 every 8 connection(s)
I20260811 22:28:19.339298 7577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:28:19.342492 7652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:19.343413 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 7577
I20260811 22:28:19.343568 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:28:19.343956 24863 external_mini_cluster.cc:1428] Setting key 89b300daf4df43f88205bda9fd267798
I20260811 22:28:19.349736 7652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Bootstrap starting.
I20260811 22:28:19.352490 7652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:19.353462 7652 log.cc:824] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:28:19 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:19.355822 7652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: No bootstrap required, opened a new log
I20260811 22:28:19.359803 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:19.345519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47764 (local address 127.24.71.254:42435)
0811 22:28:19.346063 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:19.346079 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:19.346117 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:28:19.346876 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:19.346897 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:19.347240 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:19.347619 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:19.348526 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.349619 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.350562 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.351042 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.351220 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:19.354455 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:19.354468 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:19.354485 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:19.354515 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:19.356554 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:19.357077 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:19.357082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:19.357087 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:19.357207 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:19.357514 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:19.357517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:19.357518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:19.357941 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:19.358110 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:19.358544 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 22:28:19.358921 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":155,"threads_started":1}
I20260811 22:28:19.360255 7652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:19.360543 7652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:19.360626 7652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a812062dbb4422c82d550f502772fb3, State: Initialized, Role: FOLLOWER
I20260811 22:28:19.361178 7652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:19.361354 7652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:28:19.361425 7652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:28:19.361541 7652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:19.362864 7652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:19.363194 7652 leader_election.cc:302] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a812062dbb4422c82d550f502772fb3; no voters:
I20260811 22:28:19.363507 7652 leader_election.cc:288] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:28:19.363705 7658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:28:19.364061 7658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8a812062dbb4422c82d550f502772fb3, State: Running, Role: LEADER
I20260811 22:28:19.364447 7658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:19.365172 7652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:28:19.366605 7659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a812062dbb4422c82d550f502772fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } } }
I20260811 22:28:19.366770 7659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:19.367154 7660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a812062dbb4422c82d550f502772fb3. Latest consensus state: current_term: 1 leader_uuid: "8a812062dbb4422c82d550f502772fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } } }
I20260811 22:28:19.367268 7660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:19.368394 7667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:28:19.371308 7667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:28:19.377354 7667 catalog_manager.cc:1347] Generated new cluster ID: 377ef523dad3439a94d9310d39d3c0f9
I20260811 22:28:19.377462 7667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:28:19.385948 7667 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:28:19.387521 7667 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:28:19.394584 7667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Generated new TSK 0
I20260811 22:28:19.395514 7667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:28:19.435127 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42435
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:19.591285 7680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:19.591982 7680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:19.592057 7680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:19.598408 7680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:19.598556 7680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:19.598631 7680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:28:19.606361 7680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:19.607779 7680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:19.609217 7680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:19.622087 7686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:19.622089 7685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:19.622231 7688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:19.622689 7680 server_base.cc:1034] running on GCE node
I20260811 22:28:19.623413 7680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:19.624459 7680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:19.625753 7680 hybrid_clock.cc:648] HybridClock initialized: now 1786487299625723 us; error 47 us; skew 500 ppm
Aug 11 22:28:19 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487299, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:19.629873 7680 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:28:19.631171 7680 webserver.cc:460] Webserver started at http://127.24.71.193:34603/ using document root <none> and password file <none>
I20260811 22:28:19.631963 7680 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:19.632077 7680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:19.632364 7680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:19.634907 7680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "238b142897484f0e9e8fd381711637b4"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "27ce24de4b5dc5829d7959e7650f62fd"
server_key_iv: "ed6e5bfbe97c02fb1f832cfcc3675b6f"
server_key_version: "encryptionkey@0"
I20260811 22:28:19.635629 7680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "238b142897484f0e9e8fd381711637b4"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "27ce24de4b5dc5829d7959e7650f62fd"
server_key_iv: "ed6e5bfbe97c02fb1f832cfcc3675b6f"
server_key_version: "encryptionkey@0"
I20260811 22:28:19.640684 7680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 22:28:19.644555 7697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:19.646509 7680 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:28:19.647012 7680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "238b142897484f0e9e8fd381711637b4"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "27ce24de4b5dc5829d7959e7650f62fd"
server_key_iv: "ed6e5bfbe97c02fb1f832cfcc3675b6f"
server_key_version: "encryptionkey@0"
I20260811 22:28:19.647222 7680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:19.662042 7680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:19.668110 7680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:19.668733 7680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:19.669863 7680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:19.671168 7680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:19.671381 7680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:19.671631 7680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:19.671741 7680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:19.686519 7680 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:44783
I20260811 22:28:19.686548 7822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:44783 every 8 connection(s)
I20260811 22:28:19.687738 7680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:28:19.696095 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 7680
I20260811 22:28:19.696269 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:28:19.696604 24863 external_mini_cluster.cc:1428] Setting key 0de40ef46177efa8b75373cd4f2548d7
Aug 11 22:28:19 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487299, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:19.708554 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:19.690700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:43473 (local address 127.24.71.254:42435)
0811 22:28:19.690878 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:19.690883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:19.692243 (+ 1360us) server_negotiation.cc:407] Connection header received
0811 22:28:19.693735 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:19.693739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:19.693813 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:19.693943 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:19.695574 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.696621 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.698511 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.698879 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.699030 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:19.702759 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:19.702780 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:19.702786 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:19.702831 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:19.705196 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:19.706031 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:19.706040 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:19.706043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:19.706168 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:19.706707 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:19.706714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:19.706719 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:19.707043 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:19.707165 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:19.708101 (+ 936us) server_negotiation.cc:299] Negotiation successful
0811 22:28:19.708280 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:28:19.709546 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:19.691268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:43473)
0811 22:28:19.691911 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:19.691971 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:19.693432 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:19.694134 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:19.694143 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:19.694659 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:19.695282 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:19.695302 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.696936 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.696940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:19.698280 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:19.698315 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.699131 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.699136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:19.699330 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:19.700143 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:19.700164 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:19.702494 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:19.705439 (+ 2945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:19.705447 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:19.705467 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:19.705846 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:19.706345 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:19.706348 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:19.706350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:19.706562 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:19.707302 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:19.707313 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:19.707690 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 22:28:19.708007 (+ 317us) client_negotiation.cc:240] Negotiation successful
0811 22:28:19.708424 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":161,"threads_started":1}
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:28:19.711463 7823 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42435
I20260811 22:28:19.711978 7823 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:19.712682 7823 heartbeater.cc:511] Master 127.24.71.254:42435 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
I20260811 22:28:19.715108 7613 ts_manager.cc:194] Registered new tserver with Master: 238b142897484f0e9e8fd381711637b4 (127.24.71.193:44783)
I20260811 22:28:19.717023 7613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:43473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:28:19.744534 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42435
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:19.869421 7828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:19.869758 7828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:19.869854 7828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:19.874127 7828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:19.874229 7828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:19.874322 7828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:28:19.880070 7828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:19.881264 7828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:19.882364 7828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:19.892910 7834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:19.893100 7828 server_base.cc:1034] running on GCE node
W20260811 22:28:19.892956 7837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:19.893335 7835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:19.893968 7828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:19.894717 7828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:19.895932 7828 hybrid_clock.cc:648] HybridClock initialized: now 1786487299895896 us; error 30 us; skew 500 ppm
Aug 11 22:28:19 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487299, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:19.898957 7828 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:28:19.900250 7828 webserver.cc:460] Webserver started at http://127.24.71.194:46109/ using document root <none> and password file <none>
I20260811 22:28:19.900926 7828 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:19.900996 7828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:19.901232 7828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:19.903595 7828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "a7884b56135246fc8b9cc7274ab19568"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "c553234c433eb90932ff9c78fd884900"
server_key_iv: "c5c64e757b18ab5c67f29c2d0d5a8862"
server_key_version: "encryptionkey@0"
I20260811 22:28:19.904277 7828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "a7884b56135246fc8b9cc7274ab19568"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "c553234c433eb90932ff9c78fd884900"
server_key_iv: "c5c64e757b18ab5c67f29c2d0d5a8862"
server_key_version: "encryptionkey@0"
I20260811 22:28:19.908372 7828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:28:19.911087 7845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:19.912472 7828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:19.912606 7828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "a7884b56135246fc8b9cc7274ab19568"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "c553234c433eb90932ff9c78fd884900"
server_key_iv: "c5c64e757b18ab5c67f29c2d0d5a8862"
server_key_version: "encryptionkey@0"
I20260811 22:28:19.912717 7828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:19.925637 7828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:19.928936 7828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:19.929105 7828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:19.929724 7828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:19.930547 7828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:19.930596 7828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:19.930637 7828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:19.930652 7828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:19.939551 7828 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:34065
I20260811 22:28:19.939639 7971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:34065 every 8 connection(s)
I20260811 22:28:19.940609 7828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:28:19.944025 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 7828
I20260811 22:28:19.944181 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:28:19.944469 24863 external_mini_cluster.cc:1428] Setting key ef7909666914932318d5b652d7a2632a
Aug 11 22:28:19 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487299, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:28:19.956101 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:19.942708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:37815 (local address 127.24.71.254:42435)
0811 22:28:19.942867 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:19.942871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:19.943773 (+ 902us) server_negotiation.cc:407] Connection header received
0811 22:28:19.945398 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:19.945402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:19.945460 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:19.945571 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:19.946923 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.947495 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.948789 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.949031 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.949148 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:19.951737 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:19.951740 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:19.951742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:19.951791 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:19.953673 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:19.954249 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:19.954254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:19.954256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:19.954360 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:19.954777 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:19.954783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:19.954786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:19.955057 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:19.955162 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:19.955786 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 22:28:19.955973 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:28:19.957260 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:19.943037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:37815)
0811 22:28:19.943604 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:19.943650 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:19.945166 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:19.945751 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:19.945758 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:19.946113 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:19.946649 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:19.946661 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.947642 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.947646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:19.948654 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:19.948664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:19.949191 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:19.949196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:19.949368 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:19.949997 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:19.950014 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:19.951534 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:19.953813 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:19.953821 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:19.953839 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:19.954133 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:19.954494 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:19.954496 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:19.954498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:19.954662 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:19.955169 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:19.955174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:19.955535 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 22:28:19.955720 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 22:28:19.956094 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:28:19.958942 7972 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42435
I20260811 22:28:19.959219 7972 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:19.959762 7972 heartbeater.cc:511] Master 127.24.71.254:42435 requested a full tablet report, sending...
I20260811 22:28:19.961089 7613 ts_manager.cc:194] Registered new tserver with Master: a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
I20260811 22:28:19.961771 7613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:37815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:28:19.979499 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42435
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:20.102489 7978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:20.102818 7978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:20.102864 7978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:20.107079 7978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:20.107156 7978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:20.107216 7978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:28:20.112918 7978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:20.114028 7978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:20.115041 7978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:20.125650 7987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:20.125811 7978 server_base.cc:1034] running on GCE node
W20260811 22:28:20.125663 7984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:20.125996 7985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:20.126439 7978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:20.127236 7978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:20.128482 7978 hybrid_clock.cc:648] HybridClock initialized: now 1786487300128445 us; error 49 us; skew 500 ppm
Aug 11 22:28:20 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487300, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:20.131680 7978 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:28:20.133083 7978 webserver.cc:460] Webserver started at http://127.24.71.195:35981/ using document root <none> and password file <none>
I20260811 22:28:20.133826 7978 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:20.133926 7978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:20.134241 7978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:20.136940 7978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "807b1e4077874b08a54e3468d606a9e6"
format_stamp: "Formatted at 2026-08-11 22:28:20 on dist-test-slave-mvvj"
server_key: "6ae60ae3bd9267d403569095885020c5"
server_key_iv: "0d113aaf1688123689da968adfa09e16"
server_key_version: "encryptionkey@0"
I20260811 22:28:20.137569 7978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "807b1e4077874b08a54e3468d606a9e6"
format_stamp: "Formatted at 2026-08-11 22:28:20 on dist-test-slave-mvvj"
server_key: "6ae60ae3bd9267d403569095885020c5"
server_key_iv: "0d113aaf1688123689da968adfa09e16"
server_key_version: "encryptionkey@0"
I20260811 22:28:20.141870 7978 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:28:20.144814 7994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:20.146437 7978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:20.146617 7978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "807b1e4077874b08a54e3468d606a9e6"
format_stamp: "Formatted at 2026-08-11 22:28:20 on dist-test-slave-mvvj"
server_key: "6ae60ae3bd9267d403569095885020c5"
server_key_iv: "0d113aaf1688123689da968adfa09e16"
server_key_version: "encryptionkey@0"
I20260811 22:28:20.146759 7978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:20.163586 7978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:20.167361 7978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:20.167725 7978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:20.168589 7978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:20.169519 7978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:20.169596 7978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:20.169674 7978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:20.169718 7978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:20.178860 7978 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:34109
I20260811 22:28:20.178884 8120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:34109 every 8 connection(s)
I20260811 22:28:20.180007 7978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:28:20.188329 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 7978
I20260811 22:28:20.188478 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:28:20.188810 24863 external_mini_cluster.cc:1428] Setting key 40cc20c997b84dfe297cbabfa27a0aef
Aug 11 22:28:20 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487300, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:20.194569 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.181993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:51541 (local address 127.24.71.254:42435)
0811 22:28:20.182145 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.182149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.183017 (+ 868us) server_negotiation.cc:407] Connection header received
0811 22:28:20.184463 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.184466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.184519 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.184601 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:20.185908 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.186503 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.187673 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.187934 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.188027 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.191021 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:20.191025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:20.191028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:20.191060 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:20.192663 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:20.193160 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:20.193163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:20.193164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:20.193239 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:20.193545 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:20.193547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:20.193549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:20.193727 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:20.193803 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.194322 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.194447 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:28:20.195431 8124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.182314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:51541)
0811 22:28:20.182818 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.182862 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:20.184229 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:20.184776 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:20.184783 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:20.185174 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:20.185722 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.185737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.186635 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.186638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.187546 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.187557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.188138 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.188145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.188369 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.189120 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:20.189142 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:20.190849 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:20.192775 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:20.192781 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:20.192794 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:20.193069 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:20.193319 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:20.193321 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:20.193322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:20.193465 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:20.193816 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:20.193820 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:20.194089 (+ 269us) client_negotiation.cc:769] Sending connection context
0811 22:28:20.194256 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 22:28:20.194541 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
I20260811 22:28:20.196727 8121 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42435
I20260811 22:28:20.197023 8121 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:20.197559 8121 heartbeater.cc:511] Master 127.24.71.254:42435 requested a full tablet report, sending...
I20260811 22:28:20.198809 7613 ts_manager.cc:194] Registered new tserver with Master: 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109)
I20260811 22:28:20.199441 7613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:51541
I20260811 22:28:20.203600 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:28:20.214188 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.206040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47768 (local address 127.24.71.254:42435)
0811 22:28:20.206199 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.206205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.206369 (+ 164us) server_negotiation.cc:407] Connection header received
0811 22:28:20.206525 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.206528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.206586 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.206660 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:20.207377 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.207947 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.208884 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.209112 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.209199 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.210806 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:20.210809 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:20.210811 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:20.210839 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:20.212401 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:20.212856 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:20.212859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:20.212860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:20.212963 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:20.213234 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:20.213236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:20.213237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:20.213474 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:20.213585 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.213775 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.213882 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:28:20.217772 7613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:28:20.220960 7613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:28:20.237416 8136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.231013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.24.71.193:44783)
0811 22:28:20.231487 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.231495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.231669 (+ 174us) server_negotiation.cc:407] Connection header received
0811 22:28:20.232113 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.232121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.232338 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.232563 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:20.232866 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.234272 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.236097 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.236672 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.236808 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.236996 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.237134 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.237242 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":124,"threads_started":1}
I20260811 22:28:20.237403 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.230564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:44783 (local address 127.0.0.1:43386)
0811 22:28:20.231552 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.231565 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:20.231930 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:20.232553 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:20.232559 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:20.232583 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:20.232732 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.232741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.234417 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.234420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.235932 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.235945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.236814 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.236817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.236935 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.236975 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:28:20.237103 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 22:28:20.237247 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":67,"thread_start_us":134,"threads_started":1}
I20260811 22:28:20.237736 8134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.230537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:34065 (local address 127.0.0.1:39030)
0811 22:28:20.231192 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.231244 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:20.231385 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:20.232970 (+ 1585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:20.232977 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:20.233010 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:20.233144 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.233151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.234614 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.234617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.235932 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.235945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.237254 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.237258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.237443 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.237478 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:28:20.237543 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:28:20.237603 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":119,"thread_start_us":74,"threads_started":1}
I20260811 22:28:20.238229 8139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.231769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.24.71.195:34109)
0811 22:28:20.232267 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.232281 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.232327 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:28:20.232396 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.232400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.232602 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.232771 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:20.233120 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.234003 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.235785 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.236778 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.236922 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.237096 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.237972 (+ 876us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.238070 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":156,"threads_started":1}
I20260811 22:28:20.238229 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.230645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.24.71.194:34065)
0811 22:28:20.232267 (+ 1622us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.232281 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.232306 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:28:20.232394 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.232400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.232602 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.232771 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:20.233299 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.234471 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.236070 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.236954 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.237061 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.237196 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.237950 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.238062 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1521,"thread_start_us":130,"threads_started":1}
I20260811 22:28:20.238620 8138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.231322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:34109 (local address 127.0.0.1:36894)
0811 22:28:20.231878 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.231897 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:20.232002 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:20.232755 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:20.232758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:20.232779 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:20.232880 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.232886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.234158 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.234164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.235637 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.235648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.237173 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.237177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.237366 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.237394 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:28:20.237474 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:28:20.237570 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":104,"thread_start_us":98,"threads_started":1}
I20260811 22:28:20.241186 7753 tablet_service.cc:1467] Processing CreateTablet for tablet 7dc5d34fa11742d0b0df9da8e22d54dc (DEFAULT_TABLE table=test-table [id=0ca347fa688146f29db98fa4c9da5ec2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:20.241556 8051 tablet_service.cc:1467] Processing CreateTablet for tablet 7dc5d34fa11742d0b0df9da8e22d54dc (DEFAULT_TABLE table=test-table [id=0ca347fa688146f29db98fa4c9da5ec2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:20.241556 7898 tablet_service.cc:1467] Processing CreateTablet for tablet 7dc5d34fa11742d0b0df9da8e22d54dc (DEFAULT_TABLE table=test-table [id=0ca347fa688146f29db98fa4c9da5ec2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:20.242532 7753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dc5d34fa11742d0b0df9da8e22d54dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:20.242674 7898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dc5d34fa11742d0b0df9da8e22d54dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:20.242682 8051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dc5d34fa11742d0b0df9da8e22d54dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:20.250547 8140 tablet_bootstrap.cc:492] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Bootstrap starting.
I20260811 22:28:20.251598 8141 tablet_bootstrap.cc:492] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: Bootstrap starting.
I20260811 22:28:20.252257 8142 tablet_bootstrap.cc:492] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: Bootstrap starting.
I20260811 22:28:20.253595 8140 tablet_bootstrap.cc:654] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:20.254490 8141 tablet_bootstrap.cc:654] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:20.254626 8140 log.cc:824] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:20.254968 8142 tablet_bootstrap.cc:654] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:20.255888 8142 log.cc:824] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:20.255921 8141 log.cc:824] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:20.257282 8140 tablet_bootstrap.cc:492] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: No bootstrap required, opened a new log
I20260811 22:28:20.257519 8140 ts_tablet_manager.cc:1397] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 22:28:20.258052 8141 tablet_bootstrap.cc:492] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: No bootstrap required, opened a new log
I20260811 22:28:20.258249 8141 ts_tablet_manager.cc:1397] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:28:20.261231 8142 tablet_bootstrap.cc:492] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: No bootstrap required, opened a new log
I20260811 22:28:20.261484 8142 ts_tablet_manager.cc:1397] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260811 22:28:20.262449 8140 raft_consensus.cc:348] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.262449 8141 raft_consensus.cc:348] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.262763 8141 raft_consensus.cc:374] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:20.262763 8140 raft_consensus.cc:374] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:20.262818 8141 raft_consensus.cc:729] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 807b1e4077874b08a54e3468d606a9e6, State: Initialized, Role: FOLLOWER
I20260811 22:28:20.262818 8140 raft_consensus.cc:729] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 238b142897484f0e9e8fd381711637b4, State: Initialized, Role: FOLLOWER
I20260811 22:28:20.263332 8141 consensus_queue.cc:260] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.263332 8140 consensus_queue.cc:260] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.264246 8121 heartbeater.cc:503] Master 127.24.71.254:42435 was elected leader, sending a full tablet report...
I20260811 22:28:20.264518 8141 ts_tablet_manager.cc:1428] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:28:20.264518 8140 ts_tablet_manager.cc:1428] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260811 22:28:20.264540 7823 heartbeater.cc:503] Master 127.24.71.254:42435 was elected leader, sending a full tablet report...
I20260811 22:28:20.266100 8142 raft_consensus.cc:348] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.266405 8142 raft_consensus.cc:374] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:20.266525 8142 raft_consensus.cc:729] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7884b56135246fc8b9cc7274ab19568, State: Initialized, Role: FOLLOWER
I20260811 22:28:20.267030 8142 consensus_queue.cc:260] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.267871 7972 heartbeater.cc:503] Master 127.24.71.254:42435 was elected leader, sending a full tablet report...
I20260811 22:28:20.269258 8142 ts_tablet_manager.cc:1428] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:28:20.386938 8146 raft_consensus.cc:482] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:20.387197 8146 raft_consensus.cc:504] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.388484 8146 leader_election.cc:288] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065), 238b142897484f0e9e8fd381711637b4 (127.24.71.193:44783)
I20260811 22:28:20.393491 8136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.388910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:36395 (local address 127.24.71.193:44783)
0811 22:28:20.389035 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.389040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.389299 (+ 259us) server_negotiation.cc:407] Connection header received
0811 22:28:20.389373 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.389376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.389423 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.389486 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:20.389758 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.390486 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.392136 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.392760 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.392891 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.392957 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.393277 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.393359 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:28:20.393491 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.388922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:56931 (local address 127.24.71.194:34065)
0811 22:28:20.389037 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.389041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.389057 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:28:20.389121 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.389124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.389170 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.389235 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:20.389596 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.390388 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.392133 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.392761 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.392902 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.392956 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.393277 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.393359 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:28:20.393513 8124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.388681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:34065 (local address 127.24.71.195:56931)
0811 22:28:20.388804 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.388822 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:20.388921 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:20.389279 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:20.389282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:20.389305 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:20.389435 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.389440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.390519 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.390521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.391991 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.392003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.392891 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.392894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.393094 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.393157 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 22:28:20.393239 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:28:20.393341 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":179,"spinlock_wait_cycles":2304}
I20260811 22:28:20.393524 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.388768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:44783 (local address 127.24.71.195:36395)
0811 22:28:20.389171 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.389182 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:20.389296 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:20.389528 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:20.389532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:20.389552 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:20.389649 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.389653 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.390596 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.390599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.391992 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.392003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.392891 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.392894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.393115 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.393161 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:28:20.393239 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:28:20.393341 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":148,"thread_start_us":141,"threads_started":1}
I20260811 22:28:20.394354 7920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dc5d34fa11742d0b0df9da8e22d54dc" candidate_uuid: "807b1e4077874b08a54e3468d606a9e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7884b56135246fc8b9cc7274ab19568" is_pre_election: true
I20260811 22:28:20.394346 7772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dc5d34fa11742d0b0df9da8e22d54dc" candidate_uuid: "807b1e4077874b08a54e3468d606a9e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "238b142897484f0e9e8fd381711637b4" is_pre_election: true
I20260811 22:28:20.394696 7772 raft_consensus.cc:2454] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 807b1e4077874b08a54e3468d606a9e6 in term 0.
I20260811 22:28:20.394696 7920 raft_consensus.cc:2454] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 807b1e4077874b08a54e3468d606a9e6 in term 0.
I20260811 22:28:20.395251 8000 leader_election.cc:302] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 238b142897484f0e9e8fd381711637b4, 807b1e4077874b08a54e3468d606a9e6, a7884b56135246fc8b9cc7274ab19568; no voters:
I20260811 22:28:20.395557 8146 raft_consensus.cc:2781] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:28:20.395664 8146 raft_consensus.cc:482] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:28:20.395730 8146 raft_consensus.cc:3037] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:20.396824 8146 raft_consensus.cc:504] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.397245 8146 leader_election.cc:288] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [CANDIDATE]: Term 1 election: Requested vote from peers a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065), 238b142897484f0e9e8fd381711637b4 (127.24.71.193:44783)
I20260811 22:28:20.397689 7772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dc5d34fa11742d0b0df9da8e22d54dc" candidate_uuid: "807b1e4077874b08a54e3468d606a9e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "238b142897484f0e9e8fd381711637b4"
I20260811 22:28:20.397689 7920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dc5d34fa11742d0b0df9da8e22d54dc" candidate_uuid: "807b1e4077874b08a54e3468d606a9e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7884b56135246fc8b9cc7274ab19568"
I20260811 22:28:20.397837 7920 raft_consensus.cc:3037] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:20.397871 7772 raft_consensus.cc:3037] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:20.399260 7772 raft_consensus.cc:2454] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 807b1e4077874b08a54e3468d606a9e6 in term 1.
I20260811 22:28:20.399262 7920 raft_consensus.cc:2454] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 807b1e4077874b08a54e3468d606a9e6 in term 1.
I20260811 22:28:20.399668 7998 leader_election.cc:302] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 238b142897484f0e9e8fd381711637b4, 807b1e4077874b08a54e3468d606a9e6, a7884b56135246fc8b9cc7274ab19568; no voters:
I20260811 22:28:20.399930 8146 raft_consensus.cc:2781] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:28:20.400177 8146 raft_consensus.cc:686] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 1 LEADER]: Becoming Leader. State: Replica: 807b1e4077874b08a54e3468d606a9e6, State: Running, Role: LEADER
I20260811 22:28:20.400581 8146 consensus_queue.cc:237] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.404024 7615 catalog_manager.cc:5495] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "807b1e4077874b08a54e3468d606a9e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } health_report { overall_health: HEALTHY } } }
W20260811 22:28:20.431241 8123 tablet.cc:2369] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:28:20.431736 8139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.427947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36908 (local address 127.24.71.195:34109)
0811 22:28:20.428096 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.428101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.428118 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:28:20.428215 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.428218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.428272 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.428391 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:28:20.428672 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.429186 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.430042 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.430261 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.430372 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.430688 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:28:20.431204 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:28:20.431332 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.431522 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.431585 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:28:20.437852 7920 raft_consensus.cc:1261] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [term 1 FOLLOWER]: Refusing update from remote peer 807b1e4077874b08a54e3468d606a9e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:20.437852 7772 raft_consensus.cc:1261] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Refusing update from remote peer 807b1e4077874b08a54e3468d606a9e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:20.438652 8146 consensus_queue.cc:1035] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:28:20.438879 8150 consensus_queue.cc:1035] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:28:20.439792 7824 tablet.cc:2369] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:28:20.441846 7973 tablet.cc:2369] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:28:20.447944 8155 mvcc.cc:205] Tried to move back new op lower bound from 7317451982585344000 to 7317451982443667456. Current Snapshot: MvccSnapshot[applied={T|T < 7317451982585344000}]
I20260811 22:28:20.457666 7615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:28:20.457890 7615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:28:20.460984 7615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Sending DeleteTablet for 3 replicas of tablet 7dc5d34fa11742d0b0df9da8e22d54dc
I20260811 22:28:20.461889 7898 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dc5d34fa11742d0b0df9da8e22d54dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:20 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:39030
I20260811 22:28:20.462071 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 7680
I20260811 22:28:20.462215 8051 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dc5d34fa11742d0b0df9da8e22d54dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:20 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:36894
I20260811 22:28:20.463796 8164 tablet_replica.cc:330] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: stopping tablet replica
I20260811 22:28:20.464195 8164 raft_consensus.cc:2229] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:28:20.464699 8164 raft_consensus.cc:2258] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:28:20.466285 8164 ts_tablet_manager.cc:1905] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:28:20.468091 8163 tablet_replica.cc:330] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: stopping tablet replica
I20260811 22:28:20.468626 8163 raft_consensus.cc:2229] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:28:20.469118 8163 raft_consensus.cc:2258] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:28:20.469421 8164 ts_tablet_manager.cc:1918] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:28:20.469686 8164 log.cc:1196] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/7dc5d34fa11742d0b0df9da8e22d54dc
I20260811 22:28:20.470086 8164 ts_tablet_manager.cc:1939] T 7dc5d34fa11742d0b0df9da8e22d54dc P 807b1e4077874b08a54e3468d606a9e6: Deleting consensus metadata
I20260811 22:28:20.471221 8163 ts_tablet_manager.cc:1905] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:28:20.471536 7599 catalog_manager.cc:4841] TS 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109): tablet 7dc5d34fa11742d0b0df9da8e22d54dc (table test-table [id=0ca347fa688146f29db98fa4c9da5ec2]) successfully deleted
I20260811 22:28:20.473762 8163 ts_tablet_manager.cc:1918] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:28:20.473875 8163 log.cc:1196] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/7dc5d34fa11742d0b0df9da8e22d54dc
W20260811 22:28:20.473958 7599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:44783 (error 108)
I20260811 22:28:20.474236 8163 ts_tablet_manager.cc:1939] T 7dc5d34fa11742d0b0df9da8e22d54dc P a7884b56135246fc8b9cc7274ab19568: Deleting consensus metadata
I20260811 22:28:20.475171 8138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.474843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:44783 (local address 127.0.0.1:43400)
0811 22:28:20.474994 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.475077 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:44783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:28:20.475171 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 7828
W20260811 22:28:20.485582 7599 catalog_manager.cc:4584] TS 238b142897484f0e9e8fd381711637b4 (127.24.71.193:44783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7dc5d34fa11742d0b0df9da8e22d54dc: Network error: Client connection negotiation failed: client connection to 127.24.71.193:44783: connect: Connection refused (error 111)
I20260811 22:28:20.491539 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 7978
I20260811 22:28:20.494302 8138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.491966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:34065 (local address 127.0.0.1:39032)
0811 22:28:20.494148 (+ 2182us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.494212 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.194:34065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2048}
I20260811 22:28:20.499735 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 7577
I20260811 22:28:20.510835 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42435
--webserver_interface=127.24.71.254
--webserver_port=33199
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:20.628679 8166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:20.628980 8166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:20.629027 8166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:20.633235 8166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:20.633311 8166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:20.633330 8166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:20.633348 8166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:20.633364 8166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:20.639061 8166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=33199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:20.640298 8166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:20.641414 8166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:20.658901 8176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:20.658995 8166 server_base.cc:1034] running on GCE node
W20260811 22:28:20.659037 8175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:20.658926 8178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:20.659811 8166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:20.661185 8166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:20.662367 8166 hybrid_clock.cc:648] HybridClock initialized: now 1786487300662373 us; error 56 us; skew 500 ppm
Aug 11 22:28:20 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487300, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:20.665763 8166 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:28:20.666913 8166 webserver.cc:460] Webserver started at http://127.24.71.254:33199/ using document root <none> and password file <none>
I20260811 22:28:20.667598 8166 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:20.667685 8166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:20.672228 8166 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:28:20.674813 8188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:20.676400 8166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:28:20.676561 8166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "8a812062dbb4422c82d550f502772fb3"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "a3992af0def569d2a82f9783d70c5db2"
server_key_iv: "6de755bd13a36ce46295730dcbd244a3"
server_key_version: "encryptionkey@0"
I20260811 22:28:20.677263 8166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:20.698207 8166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:20.702137 8166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:20.702394 8166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:20.711628 8166 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:42435
I20260811 22:28:20.712049 8244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:42435 every 8 connection(s)
I20260811 22:28:20.713176 8166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:28:20.718747 8245 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:28:20.719411 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 8166
I20260811 22:28:20.719969 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:44783
--local_ip_for_outbound_sockets=127.24.71.193
--tserver_master_addrs=127.24.71.254:42435
--webserver_port=34603
--webserver_interface=127.24.71.193
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:28:20.721535 8245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Bootstrap starting.
I20260811 22:28:20.740814 8245 log.cc:824] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:20.750808 8245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:28:20.751255 8245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Bootstrap complete.
I20260811 22:28:20.756050 8245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:20.756542 8245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a812062dbb4422c82d550f502772fb3, State: Initialized, Role: FOLLOWER
I20260811 22:28:20.757082 8245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:20.757243 8245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:28:20.757342 8245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:28:20.757481 8245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:28:20.759815 8245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:20.760264 8245 leader_election.cc:302] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a812062dbb4422c82d550f502772fb3; no voters:
I20260811 22:28:20.760650 8245 leader_election.cc:288] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:28:20.760752 8250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:28:20.760980 8250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 2 LEADER]: Becoming Leader. State: Replica: 8a812062dbb4422c82d550f502772fb3, State: Running, Role: LEADER
I20260811 22:28:20.761303 8250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:20.762074 8245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:28:20.762944 8252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a812062dbb4422c82d550f502772fb3. Latest consensus state: current_term: 2 leader_uuid: "8a812062dbb4422c82d550f502772fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } } }
I20260811 22:28:20.763077 8252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:20.762997 8251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8a812062dbb4422c82d550f502772fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } } }
I20260811 22:28:20.763134 8251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:20.767306 8267 catalog_manager.cc:1259] Loaded cluster ID: 377ef523dad3439a94d9310d39d3c0f9
I20260811 22:28:20.767406 8267 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: loading cluster ID for follower catalog manager: success
W20260811 22:28:20.768378 8267 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260811 22:28:20.768468 8267 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260811 22:28:20.768545 8258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:28:20.771273 8258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:28:20.771790 8258 catalog_manager.cc:1259] Loaded cluster ID: 377ef523dad3439a94d9310d39d3c0f9
I20260811 22:28:20.771870 8258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:28:20.772873 8258 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 22:28:20.777259 8258 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 22:28:20.783358 8258 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:28:20.784317 8258 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Loaded TSK: 0
I20260811 22:28:20.785627 8258 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:28:20.849340 8248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:20.849643 8248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:20.849690 8248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:20.854068 8248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:20.854142 8248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:20.854234 8248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:28:20.861625 8248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:44783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=34603
--tserver_master_addrs=127.24.71.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:20.863302 8248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:20.865057 8248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:20.877719 8275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:20.877709 8276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:20.877897 8281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:20.878690 8248 server_base.cc:1034] running on GCE node
I20260811 22:28:20.879155 8248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:20.879966 8248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:20.881178 8248 hybrid_clock.cc:648] HybridClock initialized: now 1786487300881137 us; error 48 us; skew 500 ppm
Aug 11 22:28:20 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487300, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:20.884663 8248 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:28:20.885739 8248 webserver.cc:460] Webserver started at http://127.24.71.193:34603/ using document root <none> and password file <none>
I20260811 22:28:20.886355 8248 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:20.886480 8248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:20.890908 8248 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:28:20.893350 8289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:20.894775 8248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:28:20.894922 8248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "238b142897484f0e9e8fd381711637b4"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "27ce24de4b5dc5829d7959e7650f62fd"
server_key_iv: "ed6e5bfbe97c02fb1f832cfcc3675b6f"
server_key_version: "encryptionkey@0"
I20260811 22:28:20.895519 8248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:20.922657 8248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:20.926481 8248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:20.926745 8248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:20.927544 8248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:20.928970 8297 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:28:20.933820 8248 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:28:20.933949 8248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:28:20.934041 8248 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:28:20.936033 8248 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:28:20.936107 8248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 22:28:20.936424 8297 tablet_bootstrap.cc:492] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Bootstrap starting.
I20260811 22:28:20.947678 8248 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:44783
I20260811 22:28:20.947719 8414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:44783 every 8 connection(s)
I20260811 22:28:20.948851 8248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:28:20.957466 8297 log.cc:824] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:20.959100 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 8248
I20260811 22:28:20.959619 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:34065
--local_ip_for_outbound_sockets=127.24.71.194
--tserver_master_addrs=127.24.71.254:42435
--webserver_port=46109
--webserver_interface=127.24.71.194
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:28:20 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487300, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:20.967656 8297 tablet_bootstrap.cc:492] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:28:20.968130 8297 tablet_bootstrap.cc:492] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Bootstrap complete.
I20260811 22:28:20.969622 8297 ts_tablet_manager.cc:1397] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.012s
I20260811 22:28:20.969895 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.951492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:55037)
0811 22:28:20.952176 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.952237 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:20.953541 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:20.954925 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:20.954938 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:20.955529 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:20.956222 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.956244 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.957683 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.957687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.958918 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.958932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.960121 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.960128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.960383 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.961158 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:20.961196 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:20.963028 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:20.965769 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:20.965779 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:20.965800 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:20.966223 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:20.966673 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:20.966679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:20.966684 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:20.966895 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:20.967654 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:20.967664 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:20.968053 (+ 389us) client_negotiation.cc:769] Sending connection context
0811 22:28:20.968345 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 22:28:20.968775 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":204,"threads_started":1}
I20260811 22:28:20.969945 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.951492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:55037 (local address 127.24.71.254:42435)
0811 22:28:20.952017 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.952038 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.952392 (+ 354us) server_negotiation.cc:407] Connection header received
0811 22:28:20.954091 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.954105 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.954451 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.954760 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:20.956465 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.957496 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.959087 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.959945 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.960155 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.963290 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:20.963299 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:20.963316 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:20.963361 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:20.965615 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:20.966374 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:20.966380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:20.966385 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:20.966546 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:20.967020 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:20.967023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:20.967025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:20.967514 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:20.967656 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.968500 (+ 844us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.968832 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":154,"threads_started":1}
I20260811 22:28:20.972442 8297 raft_consensus.cc:348] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.973040 8297 raft_consensus.cc:729] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 238b142897484f0e9e8fd381711637b4, State: Initialized, Role: FOLLOWER
I20260811 22:28:20.973268 8415 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42435
I20260811 22:28:20.973556 8415 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:20.973549 8297 consensus_queue.cc:260] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } } peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } }
I20260811 22:28:20.974265 8415 heartbeater.cc:511] Master 127.24.71.254:42435 requested a full tablet report, sending...
I20260811 22:28:20.974699 8297 ts_tablet_manager.cc:1428] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260811 22:28:20.977276 8208 ts_manager.cc:194] Registered new tserver with Master: 238b142897484f0e9e8fd381711637b4 (127.24.71.193:44783)
I20260811 22:28:20.981093 8208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:55037
I20260811 22:28:20.984025 8415 heartbeater.cc:503] Master 127.24.71.254:42435 was elected leader, sending a full tablet report...
Aug 11 22:28:20 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487300, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:28:20.992883 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.980548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.24.71.193:44783)
0811 22:28:20.980986 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:20.980998 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:20.981460 (+ 462us) server_negotiation.cc:407] Connection header received
0811 22:28:20.981708 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:20.981714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:20.981935 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:20.982116 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:20.982915 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.984135 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.985485 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.985958 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.986045 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.988902 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:20.988916 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:20.988938 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:20.988972 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:20.990665 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:20.991243 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:20.991248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:20.991249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:20.991331 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:20.991655 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:20.991658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:20.991660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:20.992018 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:20.992167 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:20.992554 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 22:28:20.992732 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":122,"threads_started":1}
I20260811 22:28:20.992884 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:20.980430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:44783 (local address 127.0.0.1:43410)
0811 22:28:20.981264 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:20.981357 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:20.981527 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:20.982114 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:20.982120 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:20.982638 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:20.982778 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.982787 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.984281 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.984285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.985277 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:20.985291 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:20.986094 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:20.986101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:20.986220 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:20.986828 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:20.986843 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:20.988730 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:20.990802 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:20.990813 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:20.990819 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:20.991098 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:20.991444 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:20.991448 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:20.991450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:20.991546 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:20.992152 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:20.992158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:20.992298 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 22:28:20.992451 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 22:28:20.992681 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":133,"threads_started":1}
I20260811 22:28:20.994019 8341 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dc5d34fa11742d0b0df9da8e22d54dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:20 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:43410
I20260811 22:28:20.995064 8426 tablet_replica.cc:330] stopping tablet replica
I20260811 22:28:20.995397 8426 raft_consensus.cc:2229] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:28:20.995666 8426 raft_consensus.cc:2258] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:28:20.996733 8426 ts_tablet_manager.cc:1905] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:28:21.000785 8426 ts_tablet_manager.cc:1918] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:28:21.000942 8426 log.cc:1196] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/7dc5d34fa11742d0b0df9da8e22d54dc
I20260811 22:28:21.001225 8426 ts_tablet_manager.cc:1939] T 7dc5d34fa11742d0b0df9da8e22d54dc P 238b142897484f0e9e8fd381711637b4: Deleting consensus metadata
I20260811 22:28:21.003173 8192 catalog_manager.cc:4841] TS 238b142897484f0e9e8fd381711637b4 (127.24.71.193:44783): tablet 7dc5d34fa11742d0b0df9da8e22d54dc (table test-table [id=0ca347fa688146f29db98fa4c9da5ec2]) successfully deleted
W20260811 22:28:21.088097 8420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:21.088361 8420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:21.088434 8420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:21.092664 8420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:21.092730 8420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:21.092828 8420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:28:21.098374 8420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:34065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=46109
--tserver_master_addrs=127.24.71.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:21.099583 8420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:21.100715 8420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:28:21.111125 8420 server_base.cc:1034] running on GCE node
W20260811 22:28:21.111095 8435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:21.111151 8433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:21.111488 8432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:21.112033 8420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:21.112664 8420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:21.113839 8420 hybrid_clock.cc:648] HybridClock initialized: now 1786487301113808 us; error 47 us; skew 500 ppm
Aug 11 22:28:21 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:21.116881 8420 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:28:21.117802 8420 webserver.cc:460] Webserver started at http://127.24.71.194:46109/ using document root <none> and password file <none>
I20260811 22:28:21.118425 8420 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:21.118548 8420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:21.122819 8420 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:28:21.125182 8444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:21.126618 8420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:28:21.126770 8420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "a7884b56135246fc8b9cc7274ab19568"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "c553234c433eb90932ff9c78fd884900"
server_key_iv: "c5c64e757b18ab5c67f29c2d0d5a8862"
server_key_version: "encryptionkey@0"
I20260811 22:28:21.127380 8420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:21.142019 8420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:21.145314 8420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:21.145543 8420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:21.146181 8420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:21.147074 8420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:21.147150 8420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:21.147228 8420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:21.147274 8420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:21.156534 8420 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:34065
I20260811 22:28:21.156526 8575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:34065 every 8 connection(s)
I20260811 22:28:21.157634 8420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:28:21.158812 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 8420
I20260811 22:28:21.159337 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:34109
--local_ip_for_outbound_sockets=127.24.71.195
--tserver_master_addrs=127.24.71.254:42435
--webserver_port=35981
--webserver_interface=127.24.71.195
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:28:21 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:21.173137 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.159966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:52933 (local address 127.24.71.254:42435)
0811 22:28:21.160166 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:21.160171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:21.161144 (+ 973us) server_negotiation.cc:407] Connection header received
0811 22:28:21.162751 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:21.162755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:21.162823 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:21.162932 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:21.164451 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.164976 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.166127 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.166369 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.166496 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.169208 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:21.169212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:21.169214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:21.169239 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:21.170887 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:21.171460 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:21.171462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:21.171464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:21.171547 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:21.171941 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:21.171944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:21.171946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:21.172146 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:21.172263 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:21.172801 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 22:28:21.172953 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:28:21.173998 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.160474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:52933)
0811 22:28:21.160984 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:21.161038 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:21.162526 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:21.163156 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:21.163166 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:21.163640 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:21.164275 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.164289 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.165109 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.165113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.165993 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.166003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.166538 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.166542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.166720 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.167410 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:21.167426 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:21.169022 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:21.171031 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:21.171038 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:21.171055 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:21.171329 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:21.171652 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:21.171656 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:21.171658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:21.171808 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:21.172260 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:21.172268 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:21.172543 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 22:28:21.172739 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:28:21.173101 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":142,"threads_started":1}
I20260811 22:28:21.175318 8576 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42435
I20260811 22:28:21.175603 8576 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:21.176239 8576 heartbeater.cc:511] Master 127.24.71.254:42435 requested a full tablet report, sending...
I20260811 22:28:21.177582 8208 ts_manager.cc:194] Registered new tserver with Master: a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
I20260811 22:28:21.178202 8208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:52933
W20260811 22:28:21.280885 8579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:21.281245 8579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:21.281330 8579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:21.285629 8579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:21.285691 8579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:21.285777 8579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:28:21.291211 8579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:34109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=35981
--tserver_master_addrs=127.24.71.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:21.292341 8579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:21.293428 8579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:21.303557 8585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:21.303587 8586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:21.303557 8589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:21.304206 8579 server_base.cc:1034] running on GCE node
I20260811 22:28:21.304677 8579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:21.305378 8579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:21.306586 8579 hybrid_clock.cc:648] HybridClock initialized: now 1786487301306564 us; error 31 us; skew 500 ppm
Aug 11 22:28:21 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:21.309695 8579 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:28:21.310612 8579 webserver.cc:460] Webserver started at http://127.24.71.195:35981/ using document root <none> and password file <none>
I20260811 22:28:21.311154 8579 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:21.311229 8579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:21.315418 8579 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:21.317687 8598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:21.319284 8579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:28:21.319404 8579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "807b1e4077874b08a54e3468d606a9e6"
format_stamp: "Formatted at 2026-08-11 22:28:20 on dist-test-slave-mvvj"
server_key: "6ae60ae3bd9267d403569095885020c5"
server_key_iv: "0d113aaf1688123689da968adfa09e16"
server_key_version: "encryptionkey@0"
I20260811 22:28:21.320015 8579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:21.338088 8579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:21.341521 8579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:21.341683 8579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:21.342231 8579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:21.343043 8579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:21.343094 8579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:21.343135 8579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:21.343151 8579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:21.352474 8579 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:34109
I20260811 22:28:21.352499 8726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:34109 every 8 connection(s)
I20260811 22:28:21.353502 8579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:28:21.357545 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 8579
Aug 11 22:28:21 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:21.369756 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.358244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47772 (local address 127.24.71.254:42435)
0811 22:28:21.358565 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:21.358571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:21.358748 (+ 177us) server_negotiation.cc:407] Connection header received
0811 22:28:21.358867 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:21.358871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:21.358952 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:21.359062 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:21.360051 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.360959 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.362646 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.363277 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.363424 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.365397 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:21.365400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:21.365402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:21.365426 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:21.367247 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:21.367813 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:21.367817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:21.367819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:21.368231 (+ 412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:21.368613 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:21.368616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:21.368618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:21.368871 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:21.369102 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:21.369387 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 22:28:21.369539 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":244,"thread_start_us":117,"threads_started":1}
I20260811 22:28:21.372197 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.355424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:45791 (local address 127.24.71.254:42435)
0811 22:28:21.355610 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:21.355614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:21.356481 (+ 867us) server_negotiation.cc:407] Connection header received
0811 22:28:21.358014 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:21.358017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:21.358081 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:21.358190 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:21.360079 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.360969 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.362685 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.363223 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.363330 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.367089 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:21.367095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:21.367098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:21.367144 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:21.368886 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:21.369528 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:21.369532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:21.369533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:21.369611 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:21.370651 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:21.370655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:21.370657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:21.370908 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:21.371011 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:21.371861 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 22:28:21.372022 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":77,"spinlock_wait_cycles":896}
I20260811 22:28:21.374118 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.355794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:45791)
0811 22:28:21.356320 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:21.356363 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:21.357776 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:21.358423 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:21.358435 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:21.359077 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:21.359857 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.359885 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.361138 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.361142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.362580 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.362593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.363548 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.363552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.363770 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.364515 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:21.364536 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:21.366873 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:21.369008 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:21.369014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:21.369034 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:21.369411 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:21.369723 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:21.369726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:21.369727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:21.370051 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:21.371072 (+ 1021us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:21.371080 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:21.371482 (+ 402us) client_negotiation.cc:769] Sending connection context
0811 22:28:21.372444 (+ 962us) client_negotiation.cc:240] Negotiation successful
0811 22:28:21.372885 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":114,"threads_started":1}
I20260811 22:28:21.375921 8727 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42435
I20260811 22:28:21.376277 8727 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:21.377058 8727 heartbeater.cc:511] Master 127.24.71.254:42435 requested a full tablet report, sending...
I20260811 22:28:21.378460 8208 ts_manager.cc:194] Registered new tserver with Master: 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109)
I20260811 22:28:21.379240 8208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:45791
I20260811 22:28:21.384868 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:28:21.391477 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.385649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.24.71.254:42435)
0811 22:28:21.385811 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:21.385818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:21.385994 (+ 176us) server_negotiation.cc:407] Connection header received
0811 22:28:21.386236 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:21.386240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:21.386396 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:21.386522 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:28:21.386934 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.387879 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.389238 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.389670 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.389825 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.390310 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:28:21.390999 (+ 689us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:28:21.391139 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:21.391250 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 22:28:21.391312 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:28:21.392269 8208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:28:21.397953 8208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:28:21.408026 8341 tablet_service.cc:1467] Processing CreateTablet for tablet f37c84ae02484722856b50e6d86548b7 (DEFAULT_TABLE table=test-table [id=d763f79a6dc34776bb727aafab3bfa06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:21.408727 8341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f37c84ae02484722856b50e6d86548b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:21.412191 8297 tablet_bootstrap.cc:492] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Bootstrap starting.
I20260811 22:28:21.413233 8297 tablet_bootstrap.cc:654] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:21.413468 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.406994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:34109 (local address 127.0.0.1:36918)
0811 22:28:21.407118 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:21.407134 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:21.407227 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:21.408576 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:21.408580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:21.408598 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:21.408715 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.408722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.410395 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.410398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.411685 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.411692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.412935 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.412938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.413039 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.413100 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 22:28:21.413206 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:28:21.413313 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":28}
I20260811 22:28:21.413543 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.407244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36918 (local address 127.24.71.195:34109)
0811 22:28:21.407995 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:21.408009 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:21.408035 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:28:21.408121 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:21.408127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:21.408372 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:21.408883 (+ 511us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:21.409045 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.410237 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.411875 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.412810 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.412940 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.413127 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:21.413244 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 22:28:21.413368 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":209,"threads_started":1}
I20260811 22:28:21.414588 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.407361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:34065 (local address 127.0.0.1:39034)
0811 22:28:21.407619 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:21.407645 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:21.407733 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:21.408955 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:21.408958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:21.408977 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:21.409076 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.409082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.410180 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.410186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.411663 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.411676 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.414177 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.414181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.414282 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.414310 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:28:21.414363 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 22:28:21.414440 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":21,"thread_start_us":110,"threads_started":1}
I20260811 22:28:21.414728 8297 tablet_bootstrap.cc:492] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: No bootstrap required, opened a new log
I20260811 22:28:21.414822 8297 ts_tablet_manager.cc:1397] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 22:28:21.415279 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.407785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39034 (local address 127.24.71.194:34065)
0811 22:28:21.408205 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:21.408220 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:21.408249 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:28:21.408334 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:21.408340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:21.408676 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:21.408878 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:21.409215 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.410029 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.411858 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.414030 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.414796 (+ 766us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.414991 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:21.415052 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 22:28:21.415132 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":211,"threads_started":1}
I20260811 22:28:21.415258 8297 raft_consensus.cc:348] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:21.415391 8297 raft_consensus.cc:374] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:21.415431 8297 raft_consensus.cc:729] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 238b142897484f0e9e8fd381711637b4, State: Initialized, Role: FOLLOWER
I20260811 22:28:21.415598 8297 consensus_queue.cc:260] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:21.415879 8297 ts_tablet_manager.cc:1428] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 22:28:21.417111 8652 tablet_service.cc:1467] Processing CreateTablet for tablet f37c84ae02484722856b50e6d86548b7 (DEFAULT_TABLE table=test-table [id=d763f79a6dc34776bb727aafab3bfa06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:21.418282 8652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f37c84ae02484722856b50e6d86548b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:21.418217 8503 tablet_service.cc:1467] Processing CreateTablet for tablet f37c84ae02484722856b50e6d86548b7 (DEFAULT_TABLE table=test-table [id=d763f79a6dc34776bb727aafab3bfa06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:21.419325 8503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f37c84ae02484722856b50e6d86548b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:21.427386 8742 tablet_bootstrap.cc:492] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: Bootstrap starting.
I20260811 22:28:21.430119 8743 tablet_bootstrap.cc:492] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: Bootstrap starting.
I20260811 22:28:21.430382 8742 tablet_bootstrap.cc:654] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:21.431915 8742 log.cc:824] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:21.432605 8743 tablet_bootstrap.cc:654] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:21.433477 8743 log.cc:824] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:21.434291 8742 tablet_bootstrap.cc:492] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: No bootstrap required, opened a new log
I20260811 22:28:21.434532 8742 ts_tablet_manager.cc:1397] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:28:21.435106 8743 tablet_bootstrap.cc:492] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: No bootstrap required, opened a new log
I20260811 22:28:21.435257 8743 ts_tablet_manager.cc:1397] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:28:21.437835 8742 raft_consensus.cc:348] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:21.438107 8742 raft_consensus.cc:374] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:21.437973 8743 raft_consensus.cc:348] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:21.438176 8742 raft_consensus.cc:729] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 807b1e4077874b08a54e3468d606a9e6, State: Initialized, Role: FOLLOWER
I20260811 22:28:21.438213 8743 raft_consensus.cc:374] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:21.438279 8743 raft_consensus.cc:729] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7884b56135246fc8b9cc7274ab19568, State: Initialized, Role: FOLLOWER
I20260811 22:28:21.438730 8743 consensus_queue.cc:260] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:21.438722 8742 consensus_queue.cc:260] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:21.439419 8576 heartbeater.cc:503] Master 127.24.71.254:42435 was elected leader, sending a full tablet report...
I20260811 22:28:21.439774 8743 ts_tablet_manager.cc:1428] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 22:28:21.439791 8742 ts_tablet_manager.cc:1428] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:28:21.440207 8727 heartbeater.cc:503] Master 127.24.71.254:42435 was elected leader, sending a full tablet report...
I20260811 22:28:21.441995 8421 raft_consensus.cc:482] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:21.442193 8421 raft_consensus.cc:504] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:21.443446 8421 leader_election.cc:288] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109), a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
I20260811 22:28:21.448676 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.443941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:60617 (local address 127.24.71.195:34109)
0811 22:28:21.444092 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:21.444096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:21.444111 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:28:21.444161 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:21.444164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:21.444214 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:21.444292 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:21.444682 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.445526 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.447171 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.447883 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.448000 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.448049 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:21.448488 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 22:28:21.448563 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:28:21.448676 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.444075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:44117 (local address 127.24.71.194:34065)
0811 22:28:21.444193 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:21.444198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:21.444579 (+ 381us) server_negotiation.cc:407] Connection header received
0811 22:28:21.444682 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:21.444685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:21.444734 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:21.444824 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:21.445096 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.445827 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.447091 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.447776 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.447978 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.448034 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:21.448459 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 22:28:21.448544 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:28:21.448679 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.443642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:34109 (local address 127.24.71.193:60617)
0811 22:28:21.443787 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:21.443806 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:21.443954 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:21.444362 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:21.444367 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:21.444393 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:21.444515 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.444521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.445847 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.445852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.446982 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.446993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.447994 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.448001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.448191 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.448370 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 22:28:21.448450 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:28:21.448528 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":72}
I20260811 22:28:21.448688 8748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.443673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:34065 (local address 127.24.71.193:44117)
0811 22:28:21.444471 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:21.444487 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:21.444592 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:21.444829 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:21.444833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:21.444852 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:21.444980 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.444985 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.445966 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.445973 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.446962 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:21.446972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.448009 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.448012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:21.448191 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.448312 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 22:28:21.448403 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:28:21.448516 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":93,"thread_start_us":95,"threads_started":1}
I20260811 22:28:21.449307 8526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f37c84ae02484722856b50e6d86548b7" candidate_uuid: "238b142897484f0e9e8fd381711637b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7884b56135246fc8b9cc7274ab19568" is_pre_election: true
I20260811 22:28:21.449635 8526 raft_consensus.cc:2454] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 238b142897484f0e9e8fd381711637b4 in term 0.
I20260811 22:28:21.449733 8673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f37c84ae02484722856b50e6d86548b7" candidate_uuid: "238b142897484f0e9e8fd381711637b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "807b1e4077874b08a54e3468d606a9e6" is_pre_election: true
I20260811 22:28:21.450017 8673 raft_consensus.cc:2454] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 238b142897484f0e9e8fd381711637b4 in term 0.
W20260811 22:28:21.450093 8416 tablet.cc:2369] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:28:21.450213 8292 leader_election.cc:302] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 238b142897484f0e9e8fd381711637b4, a7884b56135246fc8b9cc7274ab19568; no voters:
I20260811 22:28:21.450529 8421 raft_consensus.cc:2781] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:28:21.450627 8421 raft_consensus.cc:482] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:28:21.450677 8421 raft_consensus.cc:3037] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:21.451797 8421 raft_consensus.cc:504] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:21.452248 8421 leader_election.cc:288] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 1 election: Requested vote from peers 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109), a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
I20260811 22:28:21.452667 8526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f37c84ae02484722856b50e6d86548b7" candidate_uuid: "238b142897484f0e9e8fd381711637b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7884b56135246fc8b9cc7274ab19568"
I20260811 22:28:21.452677 8673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f37c84ae02484722856b50e6d86548b7" candidate_uuid: "238b142897484f0e9e8fd381711637b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "807b1e4077874b08a54e3468d606a9e6"
I20260811 22:28:21.452813 8526 raft_consensus.cc:3037] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:21.452828 8673 raft_consensus.cc:3037] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:21.453855 8673 raft_consensus.cc:2454] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 238b142897484f0e9e8fd381711637b4 in term 1.
I20260811 22:28:21.453930 8526 raft_consensus.cc:2454] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 238b142897484f0e9e8fd381711637b4 in term 1.
I20260811 22:28:21.454206 8294 leader_election.cc:302] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 238b142897484f0e9e8fd381711637b4, 807b1e4077874b08a54e3468d606a9e6; no voters:
I20260811 22:28:21.454401 8421 raft_consensus.cc:2781] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:28:21.454674 8421 raft_consensus.cc:686] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 LEADER]: Becoming Leader. State: Replica: 238b142897484f0e9e8fd381711637b4, State: Running, Role: LEADER
I20260811 22:28:21.455078 8421 consensus_queue.cc:237] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:21.458511 8207 catalog_manager.cc:5495] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 238b142897484f0e9e8fd381711637b4 (127.24.71.193). New cstate: current_term: 1 leader_uuid: "238b142897484f0e9e8fd381711637b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } health_report { overall_health: HEALTHY } } }
I20260811 22:28:21.475054 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.470918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43414 (local address 127.24.71.193:44783)
0811 22:28:21.471173 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:21.471178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:21.471201 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:28:21.471287 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:21.471289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:21.471349 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:21.471464 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:28:21.471887 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.472507 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.473386 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:21.473630 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:21.473723 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:21.473993 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:28:21.474624 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:28:21.474728 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:21.474871 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 22:28:21.474933 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260811 22:28:21.479394 8673 raft_consensus.cc:1261] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [term 1 FOLLOWER]: Refusing update from remote peer 238b142897484f0e9e8fd381711637b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:21.479395 8526 raft_consensus.cc:1261] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [term 1 FOLLOWER]: Refusing update from remote peer 238b142897484f0e9e8fd381711637b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:21.480216 8749 consensus_queue.cc:1035] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:28:21.480432 8421 consensus_queue.cc:1035] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:28:21.489148 8752 mvcc.cc:205] Tried to move back new op lower bound from 7317451986851344384 to 7317451986761584640. Current Snapshot: MvccSnapshot[applied={T|T < 7317451986851344384}]
I20260811 22:28:21.492149 8754 mvcc.cc:205] Tried to move back new op lower bound from 7317451986851344384 to 7317451986761584640. Current Snapshot: MvccSnapshot[applied={T|T < 7317451986851344384}]
I20260811 22:28:21.498219 8207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:28:21.498433 8207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:28:21.501189 8207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Sending DeleteTablet for 3 replicas of tablet f37c84ae02484722856b50e6d86548b7
I20260811 22:28:21.501977 8341 tablet_service.cc:1514] Processing DeleteTablet for tablet f37c84ae02484722856b50e6d86548b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:21 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:43410
I20260811 22:28:21.502039 8652 tablet_service.cc:1514] Processing DeleteTablet for tablet f37c84ae02484722856b50e6d86548b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:21 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:36918
I20260811 22:28:21.502027 8503 tablet_service.cc:1514] Processing DeleteTablet for tablet f37c84ae02484722856b50e6d86548b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:21 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:39034
I20260811 22:28:21.502314 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 8248
I20260811 22:28:21.502487 8426 tablet_replica.cc:330] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: stopping tablet replica
I20260811 22:28:21.502631 8426 raft_consensus.cc:2229] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:28:21.504235 8764 tablet_replica.cc:330] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: stopping tablet replica
I20260811 22:28:21.504439 8763 tablet_replica.cc:330] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: stopping tablet replica
I20260811 22:28:21.504608 8764 raft_consensus.cc:2229] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:28:21.504704 8763 raft_consensus.cc:2229] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:28:21.504942 8763 raft_consensus.cc:2258] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:28:21.504943 8764 raft_consensus.cc:2258] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:28:21.506748 8763 ts_tablet_manager.cc:1905] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:28:21.508720 8764 ts_tablet_manager.cc:1905] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:28:21.510846 8763 ts_tablet_manager.cc:1918] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:28:21.510968 8763 log.cc:1196] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/f37c84ae02484722856b50e6d86548b7
I20260811 22:28:21.511001 8764 ts_tablet_manager.cc:1918] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:28:21.511118 8764 log.cc:1196] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/f37c84ae02484722856b50e6d86548b7
I20260811 22:28:21.511325 8763 ts_tablet_manager.cc:1939] T f37c84ae02484722856b50e6d86548b7 P a7884b56135246fc8b9cc7274ab19568: Deleting consensus metadata
I20260811 22:28:21.511514 8764 ts_tablet_manager.cc:1939] T f37c84ae02484722856b50e6d86548b7 P 807b1e4077874b08a54e3468d606a9e6: Deleting consensus metadata
I20260811 22:28:21.512748 8191 catalog_manager.cc:4841] TS a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065): tablet f37c84ae02484722856b50e6d86548b7 (table test-table [id=d763f79a6dc34776bb727aafab3bfa06]) successfully deleted
I20260811 22:28:21.512745 8192 catalog_manager.cc:4841] TS 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109): tablet f37c84ae02484722856b50e6d86548b7 (table test-table [id=d763f79a6dc34776bb727aafab3bfa06]) successfully deleted
W20260811 22:28:21.514829 8192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:44783 (error 108)
I20260811 22:28:21.515673 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 8420
I20260811 22:28:21.516008 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:21.515625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:44783 (local address 127.0.0.1:43424)
0811 22:28:21.515782 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:21.515900 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:44783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:28:21.517699 8192 catalog_manager.cc:4584] TS 238b142897484f0e9e8fd381711637b4 (127.24.71.193:44783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f37c84ae02484722856b50e6d86548b7: Network error: Client connection negotiation failed: client connection to 127.24.71.193:44783: connect: Connection refused (error 111)
I20260811 22:28:21.524499 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 8579
I20260811 22:28:21.531993 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 8166
I20260811 22:28:21.540989 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42435
--webserver_interface=127.24.71.254
--webserver_port=33199
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:21.658927 8766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:21.659197 8766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:21.659240 8766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:21.663440 8766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:21.663517 8766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:21.663537 8766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:21.663555 8766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:21.663570 8766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:21.669415 8766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=33199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:21.670549 8766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:21.671643 8766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:21.680821 8773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:21.680871 8772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:21.680828 8775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:21.681267 8766 server_base.cc:1034] running on GCE node
I20260811 22:28:21.681905 8766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:21.683092 8766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:21.684296 8766 hybrid_clock.cc:648] HybridClock initialized: now 1786487301684241 us; error 69 us; skew 500 ppm
Aug 11 22:28:21 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:21.687592 8766 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:28:21.688776 8766 webserver.cc:460] Webserver started at http://127.24.71.254:33199/ using document root <none> and password file <none>
I20260811 22:28:21.689450 8766 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:21.689525 8766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:21.694072 8766 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:28:21.696719 8783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:21.698118 8766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:28:21.698277 8766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "8a812062dbb4422c82d550f502772fb3"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "a3992af0def569d2a82f9783d70c5db2"
server_key_iv: "6de755bd13a36ce46295730dcbd244a3"
server_key_version: "encryptionkey@0"
I20260811 22:28:21.698944 8766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:21.717759 8766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:21.721740 8766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:21.721989 8766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:21.730273 8766 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:42435
I20260811 22:28:21.730624 8846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:42435 every 8 connection(s)
I20260811 22:28:21.731528 8766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:28:21.736248 8847 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:28:21.738116 8847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Bootstrap starting.
I20260811 22:28:21.739111 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 8766
I20260811 22:28:21.739650 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:44783
--local_ip_for_outbound_sockets=127.24.71.193
--tserver_master_addrs=127.24.71.254:42435
--webserver_port=34603
--webserver_interface=127.24.71.193
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:28:21.757881 8847 log.cc:824] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:21.770334 8847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:28:21.770790 8847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3: Bootstrap complete.
I20260811 22:28:21.775216 8847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:21.775646 8847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a812062dbb4422c82d550f502772fb3, State: Initialized, Role: FOLLOWER
I20260811 22:28:21.776188 8847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:21.776319 8847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:28:21.776417 8847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:28:21.776539 8847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:28:21.778604 8847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:21.778962 8847 leader_election.cc:302] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a812062dbb4422c82d550f502772fb3; no voters:
I20260811 22:28:21.779325 8847 leader_election.cc:288] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:28:21.779417 8852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:28:21.779629 8852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [term 3 LEADER]: Becoming Leader. State: Replica: 8a812062dbb4422c82d550f502772fb3, State: Running, Role: LEADER
I20260811 22:28:21.779963 8852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } }
I20260811 22:28:21.780678 8847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:28:21.780998 8852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a812062dbb4422c82d550f502772fb3. Latest consensus state: current_term: 3 leader_uuid: "8a812062dbb4422c82d550f502772fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } } }
I20260811 22:28:21.781132 8852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:21.781555 8853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8a812062dbb4422c82d550f502772fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a812062dbb4422c82d550f502772fb3" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42435 } } }
I20260811 22:28:21.781724 8853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a812062dbb4422c82d550f502772fb3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:21.782852 8856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:28:21.788442 8856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:28:21.789155 8856 catalog_manager.cc:1259] Loaded cluster ID: 377ef523dad3439a94d9310d39d3c0f9
I20260811 22:28:21.789217 8856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 22:28:21.790342 8856 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe42e4e39ad google::LogMessage::Fail() at ??:0
@ 0x7fe42e4e688a google::LogMessage::SendToLog() at ??:0
@ 0x7fe42e4e34de google::LogMessage::Flush() at ??:0
@ 0x7fe42e4e4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe438e22fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe438e23d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe438e20579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe438e64640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe4307353d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe430735c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe4307373f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe430726512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe431d6a6db start_thread at ??:0
@ 0x7fe42cfe161f clone at ??:0
W20260811 22:28:21.883569 8849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:21.883899 8849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:21.884003 8849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:21.888280 8849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:21.888341 8849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:21.888433 8849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:28:21.894048 8849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:44783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=34603
--tserver_master_addrs=127.24.71.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:21.895350 8849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:21.896588 8849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:21.907047 8895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:21.907039 8896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:21.907039 8898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:21.907558 8849 server_base.cc:1034] running on GCE node
I20260811 22:28:21.908236 8849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:21.909171 8849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:21.910399 8849 hybrid_clock.cc:648] HybridClock initialized: now 1786487301910359 us; error 58 us; skew 500 ppm
Aug 11 22:28:21 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:21.914156 8849 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:28:21.915331 8849 webserver.cc:460] Webserver started at http://127.24.71.193:34603/ using document root <none> and password file <none>
I20260811 22:28:21.916257 8849 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:21.916363 8849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:21.922014 8849 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:28:21.925052 8906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:21.926736 8849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:28:21.926882 8849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "238b142897484f0e9e8fd381711637b4"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "27ce24de4b5dc5829d7959e7650f62fd"
server_key_iv: "ed6e5bfbe97c02fb1f832cfcc3675b6f"
server_key_version: "encryptionkey@0"
I20260811 22:28:21.927685 8849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:21.946517 8849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:21.950887 8849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:21.951113 8849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:21.951930 8849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:21.953404 8913 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:28:21.957947 8849 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:28:21.958034 8849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:28:21.958109 8849 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:28:21.960515 8849 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:28:21.960578 8849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 22:28:21.961120 8913 tablet_bootstrap.cc:492] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Bootstrap starting.
I20260811 22:28:21.971626 8849 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:44783
I20260811 22:28:21.971825 9037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:44783 every 8 connection(s)
I20260811 22:28:21.972932 8849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:28:21.978787 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 8849
I20260811 22:28:21.979264 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:34065
--local_ip_for_outbound_sockets=127.24.71.194
--tserver_master_addrs=127.24.71.254:42435
--webserver_port=46109
--webserver_interface=127.24.71.194
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:28:21.984840 8913 log.cc:824] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:21.994808 8913 tablet_bootstrap.cc:492] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:28:21.995275 8913 tablet_bootstrap.cc:492] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Bootstrap complete.
I20260811 22:28:21.996802 8913 ts_tablet_manager.cc:1397] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Time spent bootstrapping tablet: real 0.036s user 0.029s sys 0.004s
I20260811 22:28:22.000453 8913 raft_consensus.cc:348] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:22.001037 8913 raft_consensus.cc:729] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 238b142897484f0e9e8fd381711637b4, State: Initialized, Role: FOLLOWER
I20260811 22:28:22.001616 8913 consensus_queue.cc:260] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:22.002791 8913 ts_tablet_manager.cc:1428] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 22:28:22.045182 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:28:21.975320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:33323)
0811 22:28:21.975902 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:21.975969 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:21.976830 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:22.044105 (+ 67275us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":157,"threads_started":1}
W20260811 22:28:22.045871 9038 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:42435 (0 consecutive failures): Network error: Failed to ping master at 127.24.71.254:42435: Client connection negotiation failed: client connection to 127.24.71.254:42435: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:28:22.046789 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:22.046460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:53707)
0811 22:28:22.046623 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:22.046719 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 22:28:22.047511 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:22.047267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:57569)
0811 22:28:22.047389 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:22.047454 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:28:22.047710 9038 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:28:22.103102 9042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:22.103364 9042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:22.103410 9042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:22.107755 9042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:22.107820 9042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:22.107986 9042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:28:22.113567 9042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:34065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=46109
--tserver_master_addrs=127.24.71.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:22.114675 9042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:22.115793 9042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:28:22.126148 9042 server_base.cc:1034] running on GCE node
W20260811 22:28:22.126106 9050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:22.126338 9052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:22.126169 9054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:22.127043 9042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:22.127777 9042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:22.129014 9042 hybrid_clock.cc:648] HybridClock initialized: now 1786487302129003 us; error 31 us; skew 500 ppm
Aug 11 22:28:22 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487302, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:22.132090 9042 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:28:22.133209 9042 webserver.cc:460] Webserver started at http://127.24.71.194:46109/ using document root <none> and password file <none>
I20260811 22:28:22.133826 9042 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:22.133945 9042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:22.137952 9042 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:28:22.140596 9062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:22.142189 9042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:22.142294 9042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "a7884b56135246fc8b9cc7274ab19568"
format_stamp: "Formatted at 2026-08-11 22:28:19 on dist-test-slave-mvvj"
server_key: "c553234c433eb90932ff9c78fd884900"
server_key_iv: "c5c64e757b18ab5c67f29c2d0d5a8862"
server_key_version: "encryptionkey@0"
I20260811 22:28:22.142832 9042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:22.163931 9042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:22.166949 9042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:22.167147 9042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:22.167906 9042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:22.168789 9042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:22.168839 9042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:22.168882 9042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:22.168897 9042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:22.178341 9042 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:34065
I20260811 22:28:22.178371 9188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:34065 every 8 connection(s)
I20260811 22:28:22.179452 9042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:28:22.183648 9191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:22.181760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:45857)
0811 22:28:22.182260 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:22.182378 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":121,"threads_started":1}
W20260811 22:28:22.184121 9189 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:42435 (0 consecutive failures): Network error: Failed to ping master at 127.24.71.254:42435: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
I20260811 22:28:22.184788 9191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:22.184558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:41253)
0811 22:28:22.184680 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:22.184736 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 22:28:22.185416 9191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:22.185209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:44385)
0811 22:28:22.185313 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:22.185359 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:28:22.185572 9189 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:28:22.187640 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 9042
I20260811 22:28:22.188076 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:34109
--local_ip_for_outbound_sockets=127.24.71.195
--tserver_master_addrs=127.24.71.254:42435
--webserver_port=35981
--webserver_interface=127.24.71.195
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:22.303313 9192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:22.303578 9192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:22.303660 9192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:22.308027 9192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:22.308092 9192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:22.308151 9192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:28:22.313400 9192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:34109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=35981
--tserver_master_addrs=127.24.71.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:22.314548 9192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:22.315611 9192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:22.325661 9197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:22.325666 9198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:22.325754 9201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:22.325974 9192 server_base.cc:1034] running on GCE node
I20260811 22:28:22.326440 9192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:22.327071 9192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:22.328227 9192 hybrid_clock.cc:648] HybridClock initialized: now 1786487302328191 us; error 49 us; skew 500 ppm
Aug 11 22:28:22 dist-test-slave-mvvj krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487302, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:22.330957 9192 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:28:22.331946 9192 webserver.cc:460] Webserver started at http://127.24.71.195:35981/ using document root <none> and password file <none>
I20260811 22:28:22.332525 9192 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:22.332594 9192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:22.336882 9192 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:28:22.339329 9208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:22.340826 9192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:28:22.340997 9192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "807b1e4077874b08a54e3468d606a9e6"
format_stamp: "Formatted at 2026-08-11 22:28:20 on dist-test-slave-mvvj"
server_key: "6ae60ae3bd9267d403569095885020c5"
server_key_iv: "0d113aaf1688123689da968adfa09e16"
server_key_version: "encryptionkey@0"
I20260811 22:28:22.341591 9192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:22.364571 9192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:22.368235 9192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:22.368461 9192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:22.369112 9192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:22.370028 9192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:22.370100 9192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:22.370174 9192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:22.370218 9192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:22.379932 9192 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:34109
I20260811 22:28:22.379971 9334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:34109 every 8 connection(s)
I20260811 22:28:22.381026 9192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:28:22.385293 9338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:22.383367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:44015)
0811 22:28:22.383898 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:22.384070 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":111,"threads_started":1}
W20260811 22:28:22.385811 9336 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:42435 (0 consecutive failures): Network error: Failed to ping master at 127.24.71.254:42435: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
I20260811 22:28:22.385808 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 9192
I20260811 22:28:22.386550 9338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:22.386274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:48459)
0811 22:28:22.386413 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:22.386478 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 22:28:22.387311 9338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:22.387106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:32841)
0811 22:28:22.387218 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:22.387260 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:28:22.387502 9336 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:28:23.048933 9342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:23.048411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:51717)
0811 22:28:23.048748 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:23.048815 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":82,"threads_started":1}
I20260811 22:28:23.186627 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:23.186172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:56169)
0811 22:28:23.186456 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:23.186515 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
I20260811 22:28:23.245784 9344 raft_consensus.cc:482] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:23.246081 9344 raft_consensus.cc:504] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:23.247439 9344 leader_election.cc:288] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109), a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
Aug 11 22:28:23 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.195@KRBTEST.COM
Aug 11 22:28:23 dist-test-slave-mvvj krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.194@KRBTEST.COM
I20260811 22:28:23.262600 9342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:23.247678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:34065 (local address 127.24.71.193:59705)
0811 22:28:23.247909 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:23.247924 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:23.248046 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:23.250218 (+ 2172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:23.250223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:23.250656 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:23.251156 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:23.251172 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:23.252630 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:23.252633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:23.253612 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:23.253630 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:23.254379 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:23.254383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:23.254604 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:23.255315 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:23.255332 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:23.257551 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:23.260090 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:23.260093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:23.260095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:23.260325 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:23.260788 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:23.260791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:23.260792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:23.260943 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:23.261757 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:23.261762 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:23.261983 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 22:28:23.262152 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 22:28:23.262436 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":76}
I20260811 22:28:23.262630 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:23.247849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:59705 (local address 127.24.71.194:34065)
0811 22:28:23.248288 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:23.248313 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:23.248357 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:28:23.249329 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:23.249338 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:23.249818 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:23.250168 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:23.251347 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:23.252432 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:23.253763 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:23.254245 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:23.254429 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:23.257773 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:23.257783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:23.257801 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:23.257843 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:23.259975 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:23.260475 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:23.260482 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:23.260488 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:23.260668 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:23.261040 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:23.261043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:23.261044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:23.261627 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:23.261771 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:23.262257 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 22:28:23.262488 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":119,"threads_started":1}
I20260811 22:28:23.263296 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:23.247680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:34109 (local address 127.24.71.193:41565)
0811 22:28:23.248503 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:23.248517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:23.248618 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:23.250675 (+ 2057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:23.250677 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:23.250842 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:23.251157 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:23.251172 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:23.252620 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:23.252623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:23.253613 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:23.253630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:23.254379 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:23.254383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:23.254606 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:23.255358 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:23.255370 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:23.257155 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:23.259807 (+ 2652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:23.259814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:23.259852 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:23.260281 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:23.261032 (+ 751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:23.261035 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:23.261037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:23.261093 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:23.262837 (+ 1744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:23.262840 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:23.262946 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 22:28:23.263039 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:28:23.263162 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":678,"thread_start_us":96,"threads_started":1}
I20260811 22:28:23.263692 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:23.247899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:41565 (local address 127.24.71.195:34109)
0811 22:28:23.248243 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:23.248260 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:23.248621 (+ 361us) server_negotiation.cc:407] Connection header received
0811 22:28:23.249383 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:23.249390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:23.249818 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:23.250168 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:23.251338 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:23.252440 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:23.253760 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:23.254241 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:23.254433 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:23.257406 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:23.257411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:23.257425 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:23.257456 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:23.259614 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:23.260476 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:23.260482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:23.260488 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:23.260668 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:23.261195 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:23.261199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:23.261200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:23.261627 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:23.261771 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:23.263243 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0811 22:28:23.263524 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":92,"threads_started":1}
W20260811 22:28:23.265753 8909 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
W20260811 22:28:23.265909 8910 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
I20260811 22:28:23.266057 8910 leader_election.cc:302] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 238b142897484f0e9e8fd381711637b4; no voters: 807b1e4077874b08a54e3468d606a9e6, a7884b56135246fc8b9cc7274ab19568
I20260811 22:28:23.266341 9344 raft_consensus.cc:2726] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:28:23.388582 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:23.388155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:33515)
0811 22:28:23.388414 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:23.388477 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260811 22:28:24.050347 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:24.049865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:33119)
0811 22:28:24.050176 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:24.050237 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":101,"threads_started":1}
I20260811 22:28:24.187964 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:24.187420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:59289)
0811 22:28:24.187715 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:24.187773 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":95,"threads_started":1}
I20260811 22:28:24.390064 9352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:24.389563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:41515)
0811 22:28:24.389877 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:24.389945 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":81,"threads_started":1}
I20260811 22:28:25.051769 9353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:25.051335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:45023)
0811 22:28:25.051606 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:25.051670 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260811 22:28:25.189254 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:25.188833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:50205)
0811 22:28:25.189095 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:25.189157 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260811 22:28:25.391547 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:25.391046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:41887)
0811 22:28:25.391372 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:25.391448 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":102,"threads_started":1}
I20260811 22:28:25.400262 9356 raft_consensus.cc:482] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:25.400429 9356 raft_consensus.cc:504] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:25.400875 9356 leader_election.cc:288] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109), a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
W20260811 22:28:25.401875 8909 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
W20260811 22:28:25.401980 8910 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
I20260811 22:28:25.402051 8910 leader_election.cc:302] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 238b142897484f0e9e8fd381711637b4; no voters: 807b1e4077874b08a54e3468d606a9e6, a7884b56135246fc8b9cc7274ab19568
I20260811 22:28:25.402194 9356 raft_consensus.cc:2726] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:28:26.053148 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:26.052678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:36585)
0811 22:28:26.052959 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:26.053028 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":83,"threads_started":1}
I20260811 22:28:26.190732 9358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:26.190187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:49255)
0811 22:28:26.190498 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:26.190593 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":79,"threads_started":1}
I20260811 22:28:26.393080 9359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:26.392556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:35047)
0811 22:28:26.392895 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:26.392965 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":109,"threads_started":1}
I20260811 22:28:27.054895 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:27.054326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:41315)
0811 22:28:27.054718 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:27.054791 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":80,"threads_started":1}
I20260811 22:28:27.192245 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:27.191748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:56159)
0811 22:28:27.192073 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:27.192144 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":139,"threads_started":1}
I20260811 22:28:27.394374 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:27.393883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:46359)
0811 22:28:27.394202 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:27.394265 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":103,"threads_started":1}
I20260811 22:28:27.769258 9363 raft_consensus.cc:482] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:27.769423 9363 raft_consensus.cc:504] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:27.770237 9363 leader_election.cc:288] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109), a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
W20260811 22:28:27.770931 8909 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
W20260811 22:28:27.771082 8910 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
I20260811 22:28:27.771171 8910 leader_election.cc:302] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 238b142897484f0e9e8fd381711637b4; no voters: 807b1e4077874b08a54e3468d606a9e6, a7884b56135246fc8b9cc7274ab19568
I20260811 22:28:27.771375 9363 raft_consensus.cc:2726] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:28:28.056357 9364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:28.055873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:47837)
0811 22:28:28.056168 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:28.056239 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
I20260811 22:28:28.193676 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:28.193200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:46067)
0811 22:28:28.193474 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:28.193541 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260811 22:28:28.395735 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:28.395287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:56455)
0811 22:28:28.395571 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:28.395636 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
I20260811 22:28:29.057646 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:29.057193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:40019)
0811 22:28:29.057481 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:29.057545 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":77,"threads_started":1}
I20260811 22:28:29.195092 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:29.194609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:52657)
0811 22:28:29.194930 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:29.194994 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":76,"threads_started":1}
I20260811 22:28:29.397219 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:29.396688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:55721)
0811 22:28:29.397048 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:29.397118 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":82,"threads_started":1}
I20260811 22:28:29.651643 9370 raft_consensus.cc:482] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:29.651793 9370 raft_consensus.cc:504] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:29.652318 9370 leader_election.cc:288] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109), a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
W20260811 22:28:29.653316 8909 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
W20260811 22:28:29.653535 8910 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
I20260811 22:28:29.653635 8910 leader_election.cc:302] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 238b142897484f0e9e8fd381711637b4; no voters: 807b1e4077874b08a54e3468d606a9e6, a7884b56135246fc8b9cc7274ab19568
I20260811 22:28:29.653859 9370 raft_consensus.cc:2726] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:28:30.059036 9371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:30.058537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:36863)
0811 22:28:30.058860 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:30.058928 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":104,"threads_started":1}
I20260811 22:28:30.196697 9372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:30.196250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:53213)
0811 22:28:30.196533 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:30.196598 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260811 22:28:30.398787 9373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:30.398253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:36145)
0811 22:28:30.398616 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:30.398685 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":107,"threads_started":1}
I20260811 22:28:31.060617 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:31.060091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:38035)
0811 22:28:31.060386 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:31.060497 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":80,"threads_started":1}
I20260811 22:28:31.198146 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:31.197694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:52047)
0811 22:28:31.197980 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:31.198046 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260811 22:28:31.400393 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:31.399798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:60819)
0811 22:28:31.400169 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:31.400235 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":114,"threads_started":1}
I20260811 22:28:32.062131 9377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:32.061663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:53479)
0811 22:28:32.061954 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:32.062025 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":83,"threads_started":1}
I20260811 22:28:32.199414 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:32.199009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:39723)
0811 22:28:32.199259 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:32.199319 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260811 22:28:32.401719 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:32.401309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:57619)
0811 22:28:32.401571 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:32.401625 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260811 22:28:33.063668 9380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:33.063117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:51293)
0811 22:28:33.063495 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:33.063566 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260811 22:28:33.200779 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:33.200246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:56577)
0811 22:28:33.200620 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:33.200681 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":189,"threads_started":1}
I20260811 22:28:33.403081 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:33.402590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:50675)
0811 22:28:33.402912 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:33.402980 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":80,"threads_started":1}
I20260811 22:28:33.808223 9384 raft_consensus.cc:482] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:33.808383 9384 raft_consensus.cc:504] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:33.808972 9384 leader_election.cc:288] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109), a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
W20260811 22:28:33.809821 8909 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
W20260811 22:28:33.809926 8910 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
I20260811 22:28:33.810055 8910 leader_election.cc:302] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 238b142897484f0e9e8fd381711637b4; no voters: 807b1e4077874b08a54e3468d606a9e6, a7884b56135246fc8b9cc7274ab19568
I20260811 22:28:33.810238 9384 raft_consensus.cc:2726] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:28:34.065338 9385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:34.064846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:54299)
0811 22:28:34.065162 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:34.065233 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":84,"threads_started":1}
I20260811 22:28:34.202343 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:34.201743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:59529)
0811 22:28:34.202124 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:34.202199 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":104,"threads_started":1}
I20260811 22:28:34.404917 9387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:34.404198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:54205)
0811 22:28:34.404675 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:34.404758 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":120,"threads_started":1}
I20260811 22:28:35.066987 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:35.066509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:55437)
0811 22:28:35.066823 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:35.066886 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":122,"threads_started":1}
I20260811 22:28:35.203877 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:35.203275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:33299)
0811 22:28:35.203603 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:35.203669 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":106,"threads_started":1}
I20260811 22:28:35.406332 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:35.405882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:51091)
0811 22:28:35.406164 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:35.406227 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":80,"threads_started":1}
I20260811 22:28:36.068468 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:36.068002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:54165)
0811 22:28:36.068289 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:36.068352 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":78,"threads_started":1}
I20260811 22:28:36.205370 9393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:36.204747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:46147)
0811 22:28:36.205209 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:36.205272 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":100,"threads_started":1}
I20260811 22:28:36.383612 9394 raft_consensus.cc:482] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:36.383747 9394 raft_consensus.cc:504] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "807b1e4077874b08a54e3468d606a9e6" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 34109 } } peers { permanent_uuid: "a7884b56135246fc8b9cc7274ab19568" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 34065 } } peers { permanent_uuid: "238b142897484f0e9e8fd381711637b4" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 44783 } }
I20260811 22:28:36.384303 9394 leader_election.cc:288] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109), a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065)
W20260811 22:28:36.385097 8910 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 807b1e4077874b08a54e3468d606a9e6 (127.24.71.195:34109): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
W20260811 22:28:36.385248 8909 leader_election.cc:341] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a7884b56135246fc8b9cc7274ab19568 (127.24.71.194:34065): Not found: Tablet not found: f37c84ae02484722856b50e6d86548b7
I20260811 22:28:36.385360 8909 leader_election.cc:302] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 238b142897484f0e9e8fd381711637b4; no voters: 807b1e4077874b08a54e3468d606a9e6, a7884b56135246fc8b9cc7274ab19568
I20260811 22:28:36.385542 9394 raft_consensus.cc:2726] T f37c84ae02484722856b50e6d86548b7 P 238b142897484f0e9e8fd381711637b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:28:36.407761 9395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:36.407255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:51147)
0811 22:28:36.407592 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:36.407660 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":107,"threads_started":1}
I20260811 22:28:37.070101 9397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:37.069600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.193:44997)
0811 22:28:37.069911 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:37.069983 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":104,"threads_started":1}
I20260811 22:28:37.206766 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:37.206332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.194:46631)
0811 22:28:37.206605 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:37.206665 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260811 22:28:37.393760 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 8849
I20260811 22:28:37.405745 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 9042
I20260811 22:28:37.409602 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:37.409127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42435 (local address 127.24.71.195:33391)
0811 22:28:37.409420 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:37.409496 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:42435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":98,"threads_started":1}
I20260811 22:28:37.415338 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 9192
2026-08-11T22:28:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19539 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:28:37 dist-test-slave-mvvj krb5kdc[9402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:28:37 dist-test-slave-mvvj krb5kdc[9402](info): set up 2 sockets
Aug 11 22:28:37 dist-test-slave-mvvj krb5kdc[9402](info): commencing operation
krb5kdc: starting...
W20260811 22:28:38.544134 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:28:38 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:28:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:28:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:28:38.691658 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40037
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:40037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:38.812894 9420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:38.813248 9420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:38.813334 9420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:38.817634 9420 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:28:38.817744 9420 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:28:38.817770 9420 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:28:38.817795 9420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:28:38.817838 9420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:38.817883 9420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:38.817942 9420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:28:38.817984 9420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:28:38.823679 9420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:40037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:38.825001 9420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:38.826200 9420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:38.835711 9427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:38.835717 9428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:38.835764 9420 server_base.cc:1034] running on GCE node
W20260811 22:28:38.835717 9431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:38.836596 9420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:38.837810 9420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:38.839028 9420 hybrid_clock.cc:648] HybridClock initialized: now 1786487318838997 us; error 45 us; skew 500 ppm
Aug 11 22:28:38 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:38.842186 9420 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:28:38.843323 9420 webserver.cc:460] Webserver started at http://127.24.71.254:43799/ using document root <none> and password file <none>
I20260811 22:28:38.844098 9420 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:38.844202 9420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:38.844462 9420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:38.846777 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "e74eda5596af4414985e0be7f881a01b"
format_stamp: "Formatted at 2026-08-11 22:28:38 on dist-test-slave-mvvj"
server_key: "5ac51a9ed55c7e7533400cadf756dc98"
server_key_iv: "3551baaee4a9537517c12c83540ae0d9"
server_key_version: "encryptionkey@0"
I20260811 22:28:38.847404 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "e74eda5596af4414985e0be7f881a01b"
format_stamp: "Formatted at 2026-08-11 22:28:38 on dist-test-slave-mvvj"
server_key: "5ac51a9ed55c7e7533400cadf756dc98"
server_key_iv: "3551baaee4a9537517c12c83540ae0d9"
server_key_version: "encryptionkey@0"
I20260811 22:28:38.851989 9420 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 22:28:38.854944 9439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:38.856454 9420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:38.856649 9420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "e74eda5596af4414985e0be7f881a01b"
format_stamp: "Formatted at 2026-08-11 22:28:38 on dist-test-slave-mvvj"
server_key: "5ac51a9ed55c7e7533400cadf756dc98"
server_key_iv: "3551baaee4a9537517c12c83540ae0d9"
server_key_version: "encryptionkey@0"
I20260811 22:28:38.856801 9420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:38.873466 9420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:38.877350 9420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:38.877609 9420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:38.886174 9420 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:40037
I20260811 22:28:38.886593 9497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:40037 every 8 connection(s)
I20260811 22:28:38.887295 9420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:28:38.890095 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 9420
I20260811 22:28:38.890250 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:28:38.890605 24863 external_mini_cluster.cc:1428] Setting key 70ef30b4ff76545f196a2687dd7cf6b2
I20260811 22:28:38.890599 9498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:38.899396 9498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Bootstrap starting.
Aug 11 22:28:38 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:38.902730 9498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:38.903892 9498 log.cc:824] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:38.906610 9498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: No bootstrap required, opened a new log
I20260811 22:28:38.907796 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:38.892176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.24.71.254:40037)
0811 22:28:38.892640 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:38.892657 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:38.892696 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:28:38.893729 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:38.893751 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:38.894126 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:38.894434 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:38.895578 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:38.896674 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:38.898010 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:38.898687 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:38.898817 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:38.901994 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:38.902001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:38.902021 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:38.902052 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:38.904403 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:38.905023 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:38.905032 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:38.905040 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:38.905219 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:38.905594 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:38.905598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:38.905599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:38.906026 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:38.906186 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:38.906576 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 22:28:38.906932 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":149,"threads_started":1}
I20260811 22:28:38.911010 9498 raft_consensus.cc:348] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:28:38.911278 9498 raft_consensus.cc:374] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:38.911346 9498 raft_consensus.cc:729] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74eda5596af4414985e0be7f881a01b, State: Initialized, Role: FOLLOWER
I20260811 22:28:38.911921 9498 consensus_queue.cc:260] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:28:38.912118 9498 raft_consensus.cc:388] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:28:38.912199 9498 raft_consensus.cc:482] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:28:38.912322 9498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:38.913493 9498 raft_consensus.cc:504] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:28:38.913921 9498 leader_election.cc:302] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e74eda5596af4414985e0be7f881a01b; no voters:
I20260811 22:28:38.914373 9498 leader_election.cc:288] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:28:38.914500 9503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:28:38.914764 9503 raft_consensus.cc:686] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 1 LEADER]: Becoming Leader. State: Replica: e74eda5596af4414985e0be7f881a01b, State: Running, Role: LEADER
I20260811 22:28:38.915086 9503 consensus_queue.cc:237] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:28:38.915875 9498 sys_catalog.cc:564] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:28:38.916549 9505 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e74eda5596af4414985e0be7f881a01b. Latest consensus state: current_term: 1 leader_uuid: "e74eda5596af4414985e0be7f881a01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } } }
I20260811 22:28:38.916997 9505 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:38.916954 9504 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e74eda5596af4414985e0be7f881a01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } } }
I20260811 22:28:38.917117 9504 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: This master's current role is: LEADER
I20260811 22:28:38.917806 9514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:28:38.921697 9514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:28:38.927511 9514 catalog_manager.cc:1347] Generated new cluster ID: 482a836230e04775ac48ff3406c2da6b
I20260811 22:28:38.927603 9514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:28:38.939117 9514 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:28:38.940259 9514 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:28:38.951403 9514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Generated new TSK 0
I20260811 22:28:38.952272 9514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:28:38.995716 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40037
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:39.119156 9527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:39.119470 9527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:39.119518 9527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:39.123679 9527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:39.123752 9527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:39.123816 9527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:28:39.129521 9527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:39.130728 9527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:39.131857 9527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:28:39.142345 9527 server_base.cc:1034] running on GCE node
W20260811 22:28:39.142289 9533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:39.142262 9535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:39.142318 9532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:39.142974 9527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:39.143690 9527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:39.144896 9527 hybrid_clock.cc:648] HybridClock initialized: now 1786487319144873 us; error 34 us; skew 500 ppm
Aug 11 22:28:39 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:39.147790 9527 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:28:39.148795 9527 webserver.cc:460] Webserver started at http://127.24.71.193:42795/ using document root <none> and password file <none>
I20260811 22:28:39.149399 9527 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:39.149502 9527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:39.149750 9527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:39.151984 9527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "6f0c10974f854df19c689ca1b346f608"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "60faee0731f5dc380dcfd09aeff93ee1"
server_key_iv: "c6960ff9d2f7bff60ba16da17f902479"
server_key_version: "encryptionkey@0"
I20260811 22:28:39.152587 9527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "6f0c10974f854df19c689ca1b346f608"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "60faee0731f5dc380dcfd09aeff93ee1"
server_key_iv: "c6960ff9d2f7bff60ba16da17f902479"
server_key_version: "encryptionkey@0"
I20260811 22:28:39.156435 9527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:28:39.159097 9545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:39.160518 9527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:39.160759 9527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "6f0c10974f854df19c689ca1b346f608"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "60faee0731f5dc380dcfd09aeff93ee1"
server_key_iv: "c6960ff9d2f7bff60ba16da17f902479"
server_key_version: "encryptionkey@0"
I20260811 22:28:39.160898 9527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:39.200687 9527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:39.205077 9527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:39.205307 9527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:39.206180 9527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:39.207149 9527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:39.207224 9527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:39.207293 9527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:39.207326 9527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:39.217604 9527 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:36247
I20260811 22:28:39.217636 9672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:36247 every 8 connection(s)
I20260811 22:28:39.218629 9527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:28:39.224858 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 9527
I20260811 22:28:39.225050 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:28:39.225354 24863 external_mini_cluster.cc:1428] Setting key 4ad0c42d1bdff61227e5fab0c5d314cb
Aug 11 22:28:39 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:39.234656 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.220712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:35749 (local address 127.24.71.254:40037)
0811 22:28:39.220885 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.220889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.222188 (+ 1299us) server_negotiation.cc:407] Connection header received
0811 22:28:39.223183 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.223187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.223249 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.223362 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:39.224696 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.225368 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.226907 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.227257 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.227401 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.230122 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:39.230127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:39.230131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:39.230171 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:39.232087 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:39.232718 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:39.232725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:39.232729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:39.232837 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:39.233236 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:39.233243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:39.233247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:39.233571 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:39.233700 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.234250 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.234422 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:28:39.235461 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.221071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:35749)
0811 22:28:39.222002 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:39.222052 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:39.222961 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:39.223576 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:39.223583 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:39.224035 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:39.224531 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.224554 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.225505 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.225515 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.226776 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.226790 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.227419 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.227421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.227587 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.228286 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:39.228304 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:39.229927 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:39.232213 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:39.232219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:39.232233 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:39.232601 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:39.232965 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:39.232968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:39.232969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:39.233122 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:39.233694 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:39.233700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:39.233967 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 22:28:39.234152 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 22:28:39.234435 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":565,"threads_started":1}
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:28:39.237000 9673 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40037
I20260811 22:28:39.237282 9673 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:39.237835 9673 heartbeater.cc:511] Master 127.24.71.254:40037 requested a full tablet report, sending...
I20260811 22:28:39.239670 9458 ts_manager.cc:194] Registered new tserver with Master: 6f0c10974f854df19c689ca1b346f608 (127.24.71.193:36247)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
I20260811 22:28:39.241416 9458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:35749
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:28:39.259322 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40037
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:39.378593 9678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:39.378911 9678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:39.378991 9678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:39.383152 9678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:39.383260 9678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:39.383356 9678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:28:39.389042 9678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:39.390244 9678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:39.391386 9678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:39.401448 9687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:39.401533 9688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:39.401469 9690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:39.401952 9678 server_base.cc:1034] running on GCE node
I20260811 22:28:39.402382 9678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:39.403056 9678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:39.404251 9678 hybrid_clock.cc:648] HybridClock initialized: now 1786487319404222 us; error 38 us; skew 500 ppm
Aug 11 22:28:39 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:39.407577 9678 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:28:39.408586 9678 webserver.cc:460] Webserver started at http://127.24.71.194:42093/ using document root <none> and password file <none>
I20260811 22:28:39.409123 9678 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:39.409183 9678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:39.409359 9678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:39.411420 9678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "173e05ebd6e1492f96172072b84af1c8"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "de131200ac9f83254134d59fa07e91b4"
server_key_iv: "b5408fa9c2ca16e8b7f5f717184d727b"
server_key_version: "encryptionkey@0"
I20260811 22:28:39.412022 9678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "173e05ebd6e1492f96172072b84af1c8"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "de131200ac9f83254134d59fa07e91b4"
server_key_iv: "b5408fa9c2ca16e8b7f5f717184d727b"
server_key_version: "encryptionkey@0"
I20260811 22:28:39.415967 9678 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:28:39.418861 9697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:39.420351 9678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:39.420526 9678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "173e05ebd6e1492f96172072b84af1c8"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "de131200ac9f83254134d59fa07e91b4"
server_key_iv: "b5408fa9c2ca16e8b7f5f717184d727b"
server_key_version: "encryptionkey@0"
I20260811 22:28:39.420671 9678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:39.447599 9678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:39.451478 9678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:39.451723 9678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:39.452502 9678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:39.453504 9678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:39.453583 9678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:39.453666 9678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:39.453723 9678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:39.467026 9678 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:45089
I20260811 22:28:39.467051 9827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:45089 every 8 connection(s)
I20260811 22:28:39.468171 9678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:28:39.478251 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 9678
Aug 11 22:28:39 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:39.478417 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:28:39.478736 24863 external_mini_cluster.cc:1428] Setting key f439382a86b5a90f6b1effb58a54bb9e
I20260811 22:28:39.484114 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.470248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:37233 (local address 127.24.71.254:40037)
0811 22:28:39.470410 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.470415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.471386 (+ 971us) server_negotiation.cc:407] Connection header received
0811 22:28:39.472831 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.472834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.472899 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.472978 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:39.474170 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.474697 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.475955 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.476191 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.476277 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.479163 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:39.479171 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:39.479176 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:39.479213 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:39.481280 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:39.481869 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:39.481875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:39.481880 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:39.481988 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:39.482397 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:39.482403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:39.482409 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:39.482718 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:39.482836 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.483721 (+ 885us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.483969 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:28:39.485374 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.470729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:37233)
0811 22:28:39.471219 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:39.471263 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:39.472610 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:39.473153 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:39.473159 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:39.473523 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:39.474034 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.474048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.474848 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.474851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.475776 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.475786 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.476339 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.476343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.476503 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.477074 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:39.477091 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:39.478966 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:39.481452 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:39.481457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:39.481472 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:39.481754 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:39.482119 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:39.482122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:39.482123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:39.482279 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:39.482890 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:39.482899 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:39.483377 (+ 478us) client_negotiation.cc:769] Sending connection context
0811 22:28:39.483640 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 22:28:39.484110 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":123,"threads_started":1}
I20260811 22:28:39.487170 9828 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40037
I20260811 22:28:39.487515 9828 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:39.488180 9828 heartbeater.cc:511] Master 127.24.71.254:40037 requested a full tablet report, sending...
I20260811 22:28:39.489531 9458 ts_manager.cc:194] Registered new tserver with Master: 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089)
I20260811 22:28:39.490424 9458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:37233
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:28:39.513610 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40037
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:28:39.630832 9834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:28:39.631120 9834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:28:39.631167 9834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:28:39.635315 9834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:28:39.635383 9834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:28:39.635443 9834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:28:39.640972 9834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:28:39.642230 9834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:28:39.643350 9834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:28:39.653939 9839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:28:39.653910 9842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:39.653980 9834 server_base.cc:1034] running on GCE node
W20260811 22:28:39.653925 9840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:28:39.654579 9834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:28:39.655318 9834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:28:39.656502 9834 hybrid_clock.cc:648] HybridClock initialized: now 1786487319656472 us; error 43 us; skew 500 ppm
Aug 11 22:28:39 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:28:39.659539 9834 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:28:39.660526 9834 webserver.cc:460] Webserver started at http://127.24.71.195:35051/ using document root <none> and password file <none>
I20260811 22:28:39.661149 9834 fs_manager.cc:359] Metadata directory not provided
I20260811 22:28:39.661244 9834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:28:39.661492 9834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:28:39.663690 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "1d77b3ecaf514deba0d2ab52f82c8d82"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "7e50c1682733bcef7f9e50ce8549dc40"
server_key_iv: "0832abed552c1ffc954559ba808bbba3"
server_key_version: "encryptionkey@0"
I20260811 22:28:39.664317 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "1d77b3ecaf514deba0d2ab52f82c8d82"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "7e50c1682733bcef7f9e50ce8549dc40"
server_key_iv: "0832abed552c1ffc954559ba808bbba3"
server_key_version: "encryptionkey@0"
I20260811 22:28:39.669203 9834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 22:28:39.672077 9852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:39.673571 9834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:28:39.673713 9834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "1d77b3ecaf514deba0d2ab52f82c8d82"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "7e50c1682733bcef7f9e50ce8549dc40"
server_key_iv: "0832abed552c1ffc954559ba808bbba3"
server_key_version: "encryptionkey@0"
I20260811 22:28:39.673844 9834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:28:39.696790 9834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:28:39.700891 9834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:28:39.701161 9834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:28:39.701884 9834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:28:39.702883 9834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:28:39.702939 9834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:39.702986 9834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:28:39.703063 9834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:28:39.713543 9834 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:33755
I20260811 22:28:39.713562 9975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:33755 every 8 connection(s)
I20260811 22:28:39.714627 9834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:28:39.722057 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 9834
I20260811 22:28:39.722249 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:28:39.722697 24863 external_mini_cluster.cc:1428] Setting key 547aeb420d1996c555b47ae4af63f66a
Aug 11 22:28:39 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:28:39.731350 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.716927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:44301 (local address 127.24.71.254:40037)
0811 22:28:39.717119 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.717123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.718249 (+ 1126us) server_negotiation.cc:407] Connection header received
0811 22:28:39.719908 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.719911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.719989 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.720084 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:39.721458 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.722021 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.723753 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.724124 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.724236 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.727505 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:39.727510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:39.727512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:39.727543 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:39.729076 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:39.729700 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:39.729703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:39.729704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:39.729791 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:39.730199 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:39.730202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:39.730203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:39.730407 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:39.730498 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.731089 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.731212 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:28:39.732343 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.717459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:44301)
0811 22:28:39.718045 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:39.718093 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:39.719624 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:39.720301 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:39.720309 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:39.720736 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:28:39.721275 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.721289 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.722182 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.722188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.723529 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.723540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.724297 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.724300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.724593 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.725398 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:28:39.725423 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:28:39.727277 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:28:39.729209 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:39.729216 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:39.729231 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:39.729593 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:39.729888 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:28:39.729892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:28:39.729894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:28:39.730086 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:28:39.730556 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:28:39.730563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:28:39.730835 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 22:28:39.731016 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 22:28:39.731303 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":135,"threads_started":1}
I20260811 22:28:39.733724 9976 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40037
I20260811 22:28:39.733981 9976 heartbeater.cc:464] Registering TS with master...
I20260811 22:28:39.734519 9976 heartbeater.cc:511] Master 127.24.71.254:40037 requested a full tablet report, sending...
I20260811 22:28:39.735788 9458 ts_manager.cc:194] Registered new tserver with Master: 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755)
I20260811 22:28:39.736647 9458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:44301
I20260811 22:28:39.737843 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:28:39.748664 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.740195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41686 (local address 127.24.71.254:40037)
0811 22:28:39.740365 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.740370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.740649 (+ 279us) server_negotiation.cc:407] Connection header received
0811 22:28:39.740729 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.740731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.740779 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.740862 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:28:39.741669 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.742200 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.743108 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.743327 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.743419 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.745064 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:28:39.745067 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:28:39.745070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:28:39.745099 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:28:39.746601 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:39.747069 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:39.747072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:39.747073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:39.747335 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:28:39.747644 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:28:39.747647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:28:39.747648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:28:39.747982 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:28:39.748097 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.748380 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.748534 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:28:39.752175 9458 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:28:39.755441 9458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:28:39.772426 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.766429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:45089 (local address 127.0.0.1:56540)
0811 22:28:39.766687 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:39.766701 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:39.766781 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:39.767748 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:39.767750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:39.767766 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:39.767878 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.767886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.769654 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.769658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.770693 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.770702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.771744 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.771748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.771899 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.771945 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:28:39.772104 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 22:28:39.772252 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":31,"thread_start_us":117,"threads_started":1}
I20260811 22:28:39.772439 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.766924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.24.71.194:45089)
0811 22:28:39.767294 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.767305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.767325 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:28:39.767399 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.767405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.767593 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.768070 (+ 477us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:39.768245 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.769455 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.770833 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.771605 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.771741 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.771967 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.772150 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.772255 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":142,"threads_started":1}
I20260811 22:28:39.772897 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.765679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:36247 (local address 127.0.0.1:50102)
0811 22:28:39.766359 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:39.766376 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:39.766472 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:39.767516 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:39.767525 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:39.767578 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:39.767770 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.767779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.769324 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.769332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.771013 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.771027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.772325 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.772330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.772460 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.772577 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 22:28:39.772650 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:28:39.772732 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":99,"spinlock_wait_cycles":1920,"thread_start_us":134,"threads_started":1}
I20260811 22:28:39.773293 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.765575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33755 (local address 127.0.0.1:60676)
0811 22:28:39.766109 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:39.766165 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:39.766315 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:39.767524 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:39.767527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:39.767555 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:39.767996 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.768006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.769618 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.769622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.771020 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.771027 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.772325 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.772330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.772474 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.772981 (+ 507us) client_negotiation.cc:769] Sending connection context
0811 22:28:39.773072 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:28:39.773157 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":731,"thread_start_us":112,"threads_started":1}
I20260811 22:28:39.773710 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.765692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60676 (local address 127.24.71.195:33755)
0811 22:28:39.767024 (+ 1332us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.767033 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.767051 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:28:39.767148 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.767154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.767359 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.767770 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:39.768220 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.769455 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.771188 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.772158 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.772593 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.772764 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.773470 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.773576 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1234,"thread_start_us":136,"threads_started":1}
I20260811 22:28:39.773741 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.766506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50102 (local address 127.24.71.193:36247)
0811 22:28:39.766882 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.766891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.766920 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:28:39.767013 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.767020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.767290 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.768214 (+ 924us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:39.768374 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.769183 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.771189 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.772158 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.773236 (+ 1078us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.773397 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.773465 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.773568 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":124,"threads_started":1}
I20260811 22:28:39.776227 9747 tablet_service.cc:1467] Processing CreateTablet for tablet f132a62e1182471ca130e0e3696f0e35 (DEFAULT_TABLE table=test-table [id=3b0b3c84b9c94d358f65926c5e3139d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:39.776263 9904 tablet_service.cc:1467] Processing CreateTablet for tablet f132a62e1182471ca130e0e3696f0e35 (DEFAULT_TABLE table=test-table [id=3b0b3c84b9c94d358f65926c5e3139d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:39.777423 9904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f132a62e1182471ca130e0e3696f0e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:39.777475 9747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f132a62e1182471ca130e0e3696f0e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:39.777994 9596 tablet_service.cc:1467] Processing CreateTablet for tablet f132a62e1182471ca130e0e3696f0e35 (DEFAULT_TABLE table=test-table [id=3b0b3c84b9c94d358f65926c5e3139d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:28:39.779068 9596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f132a62e1182471ca130e0e3696f0e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:28:39.784749 9997 tablet_bootstrap.cc:492] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: Bootstrap starting.
I20260811 22:28:39.785755 10000 tablet_bootstrap.cc:492] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: Bootstrap starting.
I20260811 22:28:39.787544 10001 tablet_bootstrap.cc:492] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Bootstrap starting.
I20260811 22:28:39.788054 10000 tablet_bootstrap.cc:654] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:39.788062 9997 tablet_bootstrap.cc:654] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:39.788892 9997 log.cc:824] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:39.789395 10000 log.cc:824] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:39.789959 10001 tablet_bootstrap.cc:654] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Neither blocks nor log segments found. Creating new log.
I20260811 22:28:39.790645 10001 log.cc:824] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Log is configured to *not* fsync() on all Append() calls
I20260811 22:28:39.791067 9997 tablet_bootstrap.cc:492] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: No bootstrap required, opened a new log
I20260811 22:28:39.791292 9997 ts_tablet_manager.cc:1397] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:28:39.792791 10001 tablet_bootstrap.cc:492] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: No bootstrap required, opened a new log
I20260811 22:28:39.793066 10001 ts_tablet_manager.cc:1397] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:28:39.793164 10000 tablet_bootstrap.cc:492] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: No bootstrap required, opened a new log
I20260811 22:28:39.793371 10000 ts_tablet_manager.cc:1397] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 22:28:39.795743 9997 raft_consensus.cc:348] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:28:39.796147 9997 raft_consensus.cc:374] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:39.796252 9997 raft_consensus.cc:729] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173e05ebd6e1492f96172072b84af1c8, State: Initialized, Role: FOLLOWER
I20260811 22:28:39.796725 10001 raft_consensus.cc:348] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:28:39.796794 9997 consensus_queue.cc:260] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:28:39.796945 10001 raft_consensus.cc:374] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:39.797026 10001 raft_consensus.cc:729] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0c10974f854df19c689ca1b346f608, State: Initialized, Role: FOLLOWER
I20260811 22:28:39.797576 9828 heartbeater.cc:503] Master 127.24.71.254:40037 was elected leader, sending a full tablet report...
I20260811 22:28:39.797657 10000 raft_consensus.cc:348] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:28:39.798009 10000 raft_consensus.cc:374] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:28:39.798066 9997 ts_tablet_manager.cc:1428] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:28:39.798089 10000 raft_consensus.cc:729] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d77b3ecaf514deba0d2ab52f82c8d82, State: Initialized, Role: FOLLOWER
I20260811 22:28:39.798338 10001 consensus_queue.cc:260] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:28:39.798614 10000 consensus_queue.cc:260] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:28:39.799234 9673 heartbeater.cc:503] Master 127.24.71.254:40037 was elected leader, sending a full tablet report...
I20260811 22:28:39.799463 10001 ts_tablet_manager.cc:1428] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:28:39.799615 10000 ts_tablet_manager.cc:1428] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 22:28:39.799952 9976 heartbeater.cc:503] Master 127.24.71.254:40037 was elected leader, sending a full tablet report...
I20260811 22:28:39.877241 10006 raft_consensus.cc:482] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:28:39.877522 10006 raft_consensus.cc:504] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:28:39.878803 10006 leader_election.cc:288] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755), 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089)
I20260811 22:28:39.885145 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.879916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:60371 (local address 127.24.71.194:45089)
0811 22:28:39.880075 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.880078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.880258 (+ 180us) server_negotiation.cc:407] Connection header received
0811 22:28:39.880411 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.880414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.880468 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.880542 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:39.880832 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.881547 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.883492 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.884191 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.884325 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.884385 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.884896 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.885009 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:28:39.885145 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.879358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:40177 (local address 127.24.71.195:33755)
0811 22:28:39.879585 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.879590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.879763 (+ 173us) server_negotiation.cc:407] Connection header received
0811 22:28:39.880145 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.880148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.880192 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.880297 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:28:39.880641 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.881473 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.883529 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.884182 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.884324 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.884385 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.884896 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.884991 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 22:28:39.885233 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.879235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33755 (local address 127.24.71.193:40177)
0811 22:28:39.879653 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:39.879666 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:39.879905 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:39.880341 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:39.880347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:39.880368 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:39.880497 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.880502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.881631 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.881637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.883307 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.883320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.884328 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.884332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.884519 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.884732 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 22:28:39.884879 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:28:39.884991 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":240,"thread_start_us":168,"threads_started":1}
I20260811 22:28:39.885233 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.879744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:45089 (local address 127.24.71.193:60371)
0811 22:28:39.880159 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:39.880171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:28:39.880290 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:28:39.880587 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:28:39.880593 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:28:39.880614 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:28:39.880713 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.880718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.881667 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.881671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.883309 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:28:39.883320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.884328 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.884332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:28:39.884512 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.884738 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 22:28:39.884850 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:28:39.884987 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":397,"thread_start_us":128,"threads_started":1}
I20260811 22:28:39.886245 9926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f132a62e1182471ca130e0e3696f0e35" candidate_uuid: "6f0c10974f854df19c689ca1b346f608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" is_pre_election: true
I20260811 22:28:39.886243 9773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f132a62e1182471ca130e0e3696f0e35" candidate_uuid: "6f0c10974f854df19c689ca1b346f608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173e05ebd6e1492f96172072b84af1c8" is_pre_election: true
I20260811 22:28:39.886597 9926 raft_consensus.cc:2454] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f0c10974f854df19c689ca1b346f608 in term 0.
I20260811 22:28:39.886591 9773 raft_consensus.cc:2454] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f0c10974f854df19c689ca1b346f608 in term 0.
I20260811 22:28:39.887202 9548 leader_election.cc:302] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d77b3ecaf514deba0d2ab52f82c8d82, 6f0c10974f854df19c689ca1b346f608; no voters:
I20260811 22:28:39.887573 10006 raft_consensus.cc:2781] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:28:39.887686 10006 raft_consensus.cc:482] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:28:39.887744 10006 raft_consensus.cc:3037] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:39.888932 10006 raft_consensus.cc:504] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:28:39.889273 10006 leader_election.cc:288] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 1 election: Requested vote from peers 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755), 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089)
I20260811 22:28:39.889746 9773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f132a62e1182471ca130e0e3696f0e35" candidate_uuid: "6f0c10974f854df19c689ca1b346f608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173e05ebd6e1492f96172072b84af1c8"
I20260811 22:28:39.889748 9926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f132a62e1182471ca130e0e3696f0e35" candidate_uuid: "6f0c10974f854df19c689ca1b346f608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82"
I20260811 22:28:39.889904 9773 raft_consensus.cc:3037] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:39.889922 9926 raft_consensus.cc:3037] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:28:39.891023 9773 raft_consensus.cc:2454] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f0c10974f854df19c689ca1b346f608 in term 1.
I20260811 22:28:39.891342 9926 raft_consensus.cc:2454] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f0c10974f854df19c689ca1b346f608 in term 1.
I20260811 22:28:39.891444 9548 leader_election.cc:302] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173e05ebd6e1492f96172072b84af1c8, 6f0c10974f854df19c689ca1b346f608; no voters:
I20260811 22:28:39.891798 10006 raft_consensus.cc:2781] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:28:39.892148 10006 raft_consensus.cc:686] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 1 LEADER]: Becoming Leader. State: Replica: 6f0c10974f854df19c689ca1b346f608, State: Running, Role: LEADER
I20260811 22:28:39.892542 10006 consensus_queue.cc:237] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:28:39.896354 9458 catalog_manager.cc:5495] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f0c10974f854df19c689ca1b346f608 (127.24.71.193). New cstate: current_term: 1 leader_uuid: "6f0c10974f854df19c689ca1b346f608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } health_report { overall_health: UNKNOWN } } }
I20260811 22:28:39.932574 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.928724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50108 (local address 127.24.71.193:36247)
0811 22:28:39.928874 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:28:39.928879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:28:39.928896 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:28:39.929001 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:28:39.929003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:28:39.929078 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:28:39.929204 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:28:39.929474 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.929998 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.930896 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:28:39.931136 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:28:39.931251 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:28:39.931603 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:28:39.932165 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:28:39.932306 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:28:39.932349 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 22:28:39.932413 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:28:39.939289 9773 raft_consensus.cc:1261] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [term 1 FOLLOWER]: Refusing update from remote peer 6f0c10974f854df19c689ca1b346f608: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:39.939314 9926 raft_consensus.cc:1261] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 1 FOLLOWER]: Refusing update from remote peer 6f0c10974f854df19c689ca1b346f608: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:28:39.940174 10010 consensus_queue.cc:1035] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:28:39.940407 10006 consensus_queue.cc:1035] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:28:39.949213 10013 mvcc.cc:205] Tried to move back new op lower bound from 7317452062462676992 to 7317452062283264000. Current Snapshot: MvccSnapshot[applied={T|T < 7317452062462676992}]
I20260811 22:28:39.951153 10014 mvcc.cc:205] Tried to move back new op lower bound from 7317452062462676992 to 7317452062283264000. Current Snapshot: MvccSnapshot[applied={T|T < 7317452062462676992}]
I20260811 22:28:39.960359 9457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:28:39.960634 9457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:28:39.963680 9457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Sending DeleteTablet for 3 replicas of tablet f132a62e1182471ca130e0e3696f0e35
I20260811 22:28:39.964818 9747 tablet_service.cc:1514] Processing DeleteTablet for tablet f132a62e1182471ca130e0e3696f0e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:39 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:56540
I20260811 22:28:39.964815 9904 tablet_service.cc:1514] Processing DeleteTablet for tablet f132a62e1182471ca130e0e3696f0e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:39 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:60676
I20260811 22:28:39.965209 9596 tablet_service.cc:1514] Processing DeleteTablet for tablet f132a62e1182471ca130e0e3696f0e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:39 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:50102
I20260811 22:28:39.965517 10023 tablet_replica.cc:330] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: stopping tablet replica
I20260811 22:28:39.965792 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 9527
I20260811 22:28:39.968585 10024 tablet_replica.cc:330] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: stopping tablet replica
I20260811 22:28:39.969005 10024 raft_consensus.cc:2229] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:28:39.969481 10024 raft_consensus.cc:2258] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:28:39.969627 10023 raft_consensus.cc:2229] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:28:39.969954 10023 raft_consensus.cc:2258] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:28:39.971269 10024 ts_tablet_manager.cc:1905] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:28:39.971596 10023 ts_tablet_manager.cc:1905] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:28:39.974036 10024 ts_tablet_manager.cc:1918] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:28:39.974182 10024 log.cc:1196] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/f132a62e1182471ca130e0e3696f0e35
I20260811 22:28:39.974611 10024 ts_tablet_manager.cc:1939] T f132a62e1182471ca130e0e3696f0e35 P 173e05ebd6e1492f96172072b84af1c8: Deleting consensus metadata
I20260811 22:28:39.975705 10023 ts_tablet_manager.cc:1918] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:28:39.975895 10023 log.cc:1196] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/f132a62e1182471ca130e0e3696f0e35
I20260811 22:28:39.976085 9442 catalog_manager.cc:4841] TS 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089): tablet f132a62e1182471ca130e0e3696f0e35 (table test-table [id=3b0b3c84b9c94d358f65926c5e3139d5]) successfully deleted
I20260811 22:28:39.976286 10023 ts_tablet_manager.cc:1939] T f132a62e1182471ca130e0e3696f0e35 P 1d77b3ecaf514deba0d2ab52f82c8d82: Deleting consensus metadata
I20260811 22:28:39.977897 9442 catalog_manager.cc:4841] TS 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755): tablet f132a62e1182471ca130e0e3696f0e35 (table test-table [id=3b0b3c84b9c94d358f65926c5e3139d5]) successfully deleted
W20260811 22:28:39.978374 9441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:36247 (error 108)
I20260811 22:28:39.979434 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 9678
I20260811 22:28:39.979704 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:28:39.979298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:36247 (local address 127.0.0.1:50118)
0811 22:28:39.979481 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 22:28:39.979580 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:36247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 22:28:39.982434 9441 catalog_manager.cc:4584] TS 6f0c10974f854df19c689ca1b346f608 (127.24.71.193:36247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f132a62e1182471ca130e0e3696f0e35: Network error: Client connection negotiation failed: client connection to 127.24.71.193:36247: connect: Connection refused (error 111)
I20260811 22:28:39.994093 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 9834
I20260811 22:28:40.003695 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 9420
I20260811 22:29:00.015448 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40037
--webserver_interface=127.24.71.254
--webserver_port=43799
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:40037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:00.143989 10027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:00.144301 10027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:00.144344 10027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:00.148623 10027 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:29:00.148695 10027 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:29:00.148712 10027 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:29:00.148726 10027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:00.148746 10027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:00.148787 10027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:00.148820 10027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:00.148840 10027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:00.154549 10027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:40037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=43799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:00.155804 10027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:00.156970 10027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:00.171617 10034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:00.171598 10033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:00.171639 10036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:00.171972 10027 server_base.cc:1034] running on GCE node
I20260811 22:29:00.172652 10027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:00.173905 10027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:00.175102 10027 hybrid_clock.cc:648] HybridClock initialized: now 1786487340175086 us; error 36 us; skew 500 ppm
Aug 11 22:29:00 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487340, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:00.183226 10027 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:29:00.184551 10027 webserver.cc:460] Webserver started at http://127.24.71.254:43799/ using document root <none> and password file <none>
I20260811 22:29:00.185197 10027 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:00.185314 10027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:00.189821 10027 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 22:29:00.192520 10043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:00.193941 10027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:00.194111 10027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "e74eda5596af4414985e0be7f881a01b"
format_stamp: "Formatted at 2026-08-11 22:28:38 on dist-test-slave-mvvj"
server_key: "5ac51a9ed55c7e7533400cadf756dc98"
server_key_iv: "3551baaee4a9537517c12c83540ae0d9"
server_key_version: "encryptionkey@0"
I20260811 22:29:00.194806 10027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:00.215291 10027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:00.219290 10027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:00.219542 10027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:00.227926 10027 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:40037
I20260811 22:29:00.228264 10099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:40037 every 8 connection(s)
I20260811 22:29:00.229249 10027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:00.234172 10100 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:29:00.235452 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 10027
I20260811 22:29:00.236038 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:36247
--local_ip_for_outbound_sockets=127.24.71.193
--tserver_master_addrs=127.24.71.254:40037
--webserver_port=42795
--webserver_interface=127.24.71.193
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:29:00.236585 10100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Bootstrap starting.
I20260811 22:29:00.256500 10100 log.cc:824] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:00.266011 10100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:29:00.266448 10100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Bootstrap complete.
I20260811 22:29:00.270740 10100 raft_consensus.cc:348] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:29:00.271190 10100 raft_consensus.cc:729] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74eda5596af4414985e0be7f881a01b, State: Initialized, Role: FOLLOWER
I20260811 22:29:00.271867 10100 consensus_queue.cc:260] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:29:00.272048 10100 raft_consensus.cc:388] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:00.272152 10100 raft_consensus.cc:482] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:00.272279 10100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:29:00.274387 10100 raft_consensus.cc:504] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:29:00.274780 10100 leader_election.cc:302] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e74eda5596af4414985e0be7f881a01b; no voters:
I20260811 22:29:00.275157 10100 leader_election.cc:288] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:29:00.275256 10106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:29:00.275480 10106 raft_consensus.cc:686] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 2 LEADER]: Becoming Leader. State: Replica: e74eda5596af4414985e0be7f881a01b, State: Running, Role: LEADER
I20260811 22:29:00.275905 10106 consensus_queue.cc:237] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:29:00.276650 10100 sys_catalog.cc:564] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:00.277727 10109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e74eda5596af4414985e0be7f881a01b. Latest consensus state: current_term: 2 leader_uuid: "e74eda5596af4414985e0be7f881a01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } } }
I20260811 22:29:00.277872 10109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:00.278278 10116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:00.278337 10108 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e74eda5596af4414985e0be7f881a01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } } }
I20260811 22:29:00.278461 10108 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:00.283943 10116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:00.285245 10116 catalog_manager.cc:1259] Loaded cluster ID: 482a836230e04775ac48ff3406c2da6b
I20260811 22:29:00.286005 10116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:00.288542 10116 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:00.289227 10116 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Loaded TSK: 0
I20260811 22:29:00.299326 10116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Generated new TSK 1
I20260811 22:29:00.299649 10116 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:29:00.373147 10103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:00.373533 10103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:00.373631 10103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:00.378567 10103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:00.378659 10103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:00.378752 10103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:29:00.385378 10103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:36247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=42795
--tserver_master_addrs=127.24.71.254:40037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:00.386562 10103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:00.387691 10103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:00.399950 10138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:00.399943 10135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:00.399943 10133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:00.401000 10103 server_base.cc:1034] running on GCE node
I20260811 22:29:00.401533 10103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:00.402329 10103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:00.403548 10103 hybrid_clock.cc:648] HybridClock initialized: now 1786487340403530 us; error 33 us; skew 500 ppm
Aug 11 22:29:00 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487340, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:00.406751 10103 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:29:00.407924 10103 webserver.cc:460] Webserver started at http://127.24.71.193:42795/ using document root <none> and password file <none>
I20260811 22:29:00.408550 10103 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:00.408634 10103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:00.412990 10103 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:29:00.415503 10146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:00.417160 10103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:29:00.417344 10103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "6f0c10974f854df19c689ca1b346f608"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "60faee0731f5dc380dcfd09aeff93ee1"
server_key_iv: "c6960ff9d2f7bff60ba16da17f902479"
server_key_version: "encryptionkey@0"
I20260811 22:29:00.418006 10103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:00.446310 10103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:00.450207 10103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:00.450455 10103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:00.451159 10103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:00.452545 10155 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:29:00.457335 10103 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:29:00.457511 10103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:29:00.457620 10103 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:29:00.459609 10103 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:29:00.459697 10103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 22:29:00.460078 10155 tablet_bootstrap.cc:492] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Bootstrap starting.
I20260811 22:29:00.471071 10103 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:36247
I20260811 22:29:00.471158 10272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:36247 every 8 connection(s)
I20260811 22:29:00.472393 10103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:29:00.476188 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 10103
I20260811 22:29:00.476866 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:45089
--local_ip_for_outbound_sockets=127.24.71.194
--tserver_master_addrs=127.24.71.254:40037
--webserver_port=42093
--webserver_interface=127.24.71.194
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:29:00.480602 10155 log.cc:824] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:29:00 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487340, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:00.489475 10155 tablet_bootstrap.cc:492] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:29:00.490022 10155 tablet_bootstrap.cc:492] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Bootstrap complete.
I20260811 22:29:00.491372 10155 ts_tablet_manager.cc:1397] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.008s
I20260811 22:29:00.493534 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.474702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:39445 (local address 127.24.71.254:40037)
0811 22:29:00.475125 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:00.475151 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:00.475553 (+ 402us) server_negotiation.cc:407] Connection header received
0811 22:29:00.476954 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:00.476971 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:00.477552 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:00.477909 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:00.479519 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.481094 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.482617 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.483472 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.483700 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.487058 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:00.487068 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:00.487089 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:00.487136 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:00.489608 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.490289 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.490299 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.490307 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.490496 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.490954 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.490961 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.490965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.491510 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:00.491661 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:00.492318 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 22:29:00.492634 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":138,"threads_started":1}
I20260811 22:29:00.493567 10277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.474751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:39445)
0811 22:29:00.475361 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:00.475410 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:00.476461 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:00.477972 (+ 1511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:00.477980 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:00.478573 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:00.479288 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:00.479314 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.481291 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.481293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:00.482451 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:00.482463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.483660 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.483665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:00.483898 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.484662 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:00.484685 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:00.486784 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:00.489749 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:00.489758 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:00.489776 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:00.490146 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:00.490626 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:00.490630 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:00.490631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:00.490834 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:00.491651 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:00.491660 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:00.491970 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 22:29:00.492174 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 22:29:00.492537 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":188,"threads_started":1}
I20260811 22:29:00.494547 10155 raft_consensus.cc:348] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:29:00.494977 10155 raft_consensus.cc:729] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0c10974f854df19c689ca1b346f608, State: Initialized, Role: FOLLOWER
I20260811 22:29:00.495479 10155 consensus_queue.cc:260] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } }
I20260811 22:29:00.496479 10155 ts_tablet_manager.cc:1428] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
I20260811 22:29:00.496538 10273 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40037
I20260811 22:29:00.496762 10273 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:00.497222 10273 heartbeater.cc:511] Master 127.24.71.254:40037 requested a full tablet report, sending...
I20260811 22:29:00.499969 10062 ts_manager.cc:194] Registered new tserver with Master: 6f0c10974f854df19c689ca1b346f608 (127.24.71.193:36247)
I20260811 22:29:00.504295 10062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:39445
I20260811 22:29:00.507479 10273 heartbeater.cc:503] Master 127.24.71.254:40037 was elected leader, sending a full tablet report...
Aug 11 22:29:00 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487340, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.193@KRBTEST.COM
I20260811 22:29:00.515452 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.503619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:36247 (local address 127.0.0.1:58600)
0811 22:29:00.504248 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:00.504296 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:00.504441 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:00.504944 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:00.504950 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:00.505436 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:00.505550 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:00.505557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.506935 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.506939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:00.507936 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:00.507950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.508585 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.508589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:00.508711 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.509302 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:00.509317 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:00.510854 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:00.513176 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:00.513186 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:00.513191 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:00.513657 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:00.514111 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:00.514114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:00.514116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:00.514217 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:00.514815 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:00.514821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:00.514976 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 22:29:00.515112 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 22:29:00.515306 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":158,"threads_started":1}
I20260811 22:29:00.515581 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.503641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58600 (local address 127.24.71.193:36247)
0811 22:29:00.504496 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:00.504523 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:00.504548 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:29:00.504625 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:00.504629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:00.504788 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:00.504940 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:00.505721 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.506791 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.508073 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.508455 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.508583 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.511004 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:00.511015 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:00.511028 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:00.511065 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:00.513018 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.513865 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.513872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.513874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.513980 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.514325 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.514328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.514329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.514649 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:00.514788 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:00.515225 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 22:29:00.515410 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":752,"thread_start_us":152,"threads_started":1}
I20260811 22:29:00.517194 10198 tablet_service.cc:1514] Processing DeleteTablet for tablet f132a62e1182471ca130e0e3696f0e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:28:39 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58600
I20260811 22:29:00.518271 10283 tablet_replica.cc:330] stopping tablet replica
I20260811 22:29:00.518688 10283 raft_consensus.cc:2229] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:00.518991 10283 raft_consensus.cc:2258] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:00.520578 10283 ts_tablet_manager.cc:1905] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:00.525671 10283 ts_tablet_manager.cc:1918] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:00.525792 10283 log.cc:1196] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal/wals/f132a62e1182471ca130e0e3696f0e35
I20260811 22:29:00.526108 10283 ts_tablet_manager.cc:1939] T f132a62e1182471ca130e0e3696f0e35 P 6f0c10974f854df19c689ca1b346f608: Deleting consensus metadata
I20260811 22:29:00.528566 10045 catalog_manager.cc:4841] TS 6f0c10974f854df19c689ca1b346f608 (127.24.71.193:36247): tablet f132a62e1182471ca130e0e3696f0e35 (table test-table [id=3b0b3c84b9c94d358f65926c5e3139d5]) successfully deleted
W20260811 22:29:00.613911 10278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:00.614272 10278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:00.614369 10278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:00.619089 10278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:00.619221 10278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:00.619328 10278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:29:00.625295 10278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:45089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=42093
--tserver_master_addrs=127.24.71.254:40037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:00.626575 10278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:00.627753 10278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:00.638425 10288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:00.638444 10293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:00.638700 10278 server_base.cc:1034] running on GCE node
W20260811 22:29:00.638425 10289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:00.639346 10278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:00.640151 10278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:00.641327 10278 hybrid_clock.cc:648] HybridClock initialized: now 1786487340641272 us; error 72 us; skew 500 ppm
Aug 11 22:29:00 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487340, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:00.644502 10278 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:29:00.645533 10278 webserver.cc:460] Webserver started at http://127.24.71.194:42093/ using document root <none> and password file <none>
I20260811 22:29:00.646162 10278 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:00.646282 10278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:00.650712 10278 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:29:00.653416 10301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:00.654816 10278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:29:00.654999 10278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "173e05ebd6e1492f96172072b84af1c8"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "de131200ac9f83254134d59fa07e91b4"
server_key_iv: "b5408fa9c2ca16e8b7f5f717184d727b"
server_key_version: "encryptionkey@0"
I20260811 22:29:00.655629 10278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:00.672633 10278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:00.676990 10278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:00.677241 10278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:00.678341 10278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:00.679524 10278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:00.679625 10278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:00.679716 10278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:00.679778 10278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:00.690536 10278 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:45089
I20260811 22:29:00.690606 10426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:45089 every 8 connection(s)
I20260811 22:29:00.691757 10278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:29:00.697286 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 10278
I20260811 22:29:00.697798 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:33755
--local_ip_for_outbound_sockets=127.24.71.195
--tserver_master_addrs=127.24.71.254:40037
--webserver_port=35051
--webserver_interface=127.24.71.195
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:29:00 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487340, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:00.708564 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.694034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:46035 (local address 127.24.71.254:40037)
0811 22:29:00.694226 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:00.694231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:00.695506 (+ 1275us) server_negotiation.cc:407] Connection header received
0811 22:29:00.696488 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:00.696492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:00.696548 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:00.696629 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:00.698223 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.698772 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.700158 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.700410 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.700541 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.703874 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:00.703880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:00.703883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:00.703923 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:00.705748 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.706542 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.706546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.706548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.706655 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.707132 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.707137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.707139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.707359 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:00.707488 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:00.708211 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 22:29:00.708385 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:29:00.709547 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.694494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:46035)
0811 22:29:00.695278 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:00.695336 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:00.696235 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:00.696828 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:00.696836 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:00.697250 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:00.697931 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:00.698028 (+ 97us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.698954 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.698957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:00.700013 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:00.700028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.700596 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.700600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:00.700789 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.701474 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:00.701493 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:00.703566 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:00.705927 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:00.705936 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:00.705960 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:00.706298 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:00.706790 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:00.706795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:00.706797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:00.707008 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:00.707522 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:00.707529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:00.707921 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 22:29:00.708165 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 22:29:00.708581 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":139,"threads_started":1}
I20260811 22:29:00.711293 10427 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40037
I20260811 22:29:00.711687 10427 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:00.712357 10427 heartbeater.cc:511] Master 127.24.71.254:40037 requested a full tablet report, sending...
I20260811 22:29:00.713824 10062 ts_manager.cc:194] Registered new tserver with Master: 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089)
I20260811 22:29:00.714489 10062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:46035
W20260811 22:29:00.833685 10431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:00.834054 10431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:00.834142 10431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:00.839191 10431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:00.839280 10431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:00.839352 10431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:29:00.845577 10431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:33755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=35051
--tserver_master_addrs=127.24.71.254:40037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:00.847076 10431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:00.848500 10431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:00.860167 10437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:00.860251 10436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:00.860240 10441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:00.860630 10431 server_base.cc:1034] running on GCE node
I20260811 22:29:00.861140 10431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:00.861869 10431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:00.863073 10431 hybrid_clock.cc:648] HybridClock initialized: now 1786487340863049 us; error 38 us; skew 500 ppm
Aug 11 22:29:00 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487340, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:00.866660 10431 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:29:00.868081 10431 webserver.cc:460] Webserver started at http://127.24.71.195:35051/ using document root <none> and password file <none>
I20260811 22:29:00.868852 10431 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:00.869004 10431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:00.873999 10431 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 22:29:00.876873 10450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:00.878729 10431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:29:00.878895 10431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "1d77b3ecaf514deba0d2ab52f82c8d82"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "7e50c1682733bcef7f9e50ce8549dc40"
server_key_iv: "0832abed552c1ffc954559ba808bbba3"
server_key_version: "encryptionkey@0"
I20260811 22:29:00.879540 10431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:00.938659 10431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:00.942853 10431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:00.943099 10431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:00.943821 10431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:00.944869 10431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:00.944922 10431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:00.944967 10431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:00.944983 10431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:00.957517 10431 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:33755
I20260811 22:29:00.957587 10581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:33755 every 8 connection(s)
I20260811 22:29:00.958755 10431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:29:00.963090 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 10431
Aug 11 22:29:00 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487340, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:00.975471 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.963859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.24.71.254:40037)
0811 22:29:00.964153 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:00.964159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:00.964368 (+ 209us) server_negotiation.cc:407] Connection header received
0811 22:29:00.964476 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:00.964479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:00.964539 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:00.964655 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:00.966304 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.967127 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.968228 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.968535 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.968623 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.970624 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:00.970631 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:00.970638 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:00.970685 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:00.972974 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.973856 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.973862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.973864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.973969 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.974317 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.974321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.974323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.974548 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:00.974794 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:00.974981 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 22:29:00.975161 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":102,"threads_started":1}
I20260811 22:29:00.977063 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.961125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:36701 (local address 127.24.71.254:40037)
0811 22:29:00.961295 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:00.961299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:00.962443 (+ 1144us) server_negotiation.cc:407] Connection header received
0811 22:29:00.963575 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:00.963579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:00.963653 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:00.963749 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:00.965517 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.966127 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.967904 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.968159 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.968302 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.972102 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:00.972107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:00.972113 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:00.972170 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:00.974332 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.975024 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.975028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.975030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.975121 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:00.975539 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:00.975543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:00.975545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:00.975799 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:00.975960 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:00.976723 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 22:29:00.976881 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:29:00.978241 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.961429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:36701)
0811 22:29:00.962263 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:00.962316 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:00.963311 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:00.964074 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:00.964086 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:00.964709 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:00.965332 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:00.965353 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.966298 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.966303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:00.967687 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:00.967699 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.968355 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.968359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:00.968531 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.969553 (+ 1022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:00.969583 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:00.971812 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:00.974491 (+ 2679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:00.974500 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:00.974526 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:00.974890 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:00.975231 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:00.975235 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:00.975237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:00.975416 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:00.975964 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:00.975973 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:00.976427 (+ 454us) client_negotiation.cc:769] Sending connection context
0811 22:29:00.976625 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 22:29:00.977035 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":115,"threads_started":1}
I20260811 22:29:00.979681 10583 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40037
I20260811 22:29:00.980050 10583 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:00.980671 10583 heartbeater.cc:511] Master 127.24.71.254:40037 requested a full tablet report, sending...
I20260811 22:29:00.982151 10062 ts_manager.cc:194] Registered new tserver with Master: 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755)
I20260811 22:29:00.982867 10062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:36701
I20260811 22:29:00.990715 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:29:00.995724 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.991530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.24.71.254:40037)
0811 22:29:00.991671 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:00.991676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:00.991803 (+ 127us) server_negotiation.cc:407] Connection header received
0811 22:29:00.992060 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:00.992063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:00.992182 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:00.992276 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:29:00.992690 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.993251 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.994252 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.994607 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.994744 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:00.995163 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:29:00.995337 (+ 174us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 22:29:00.995598 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34852: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 22:29:00.995893 10275 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34852: token expired
W20260811 22:29:00.996407 9981 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.71.254:40037)
I20260811 22:29:01.005672 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:00.996950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34862 (local address 127.24.71.254:40037)
0811 22:29:00.997086 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:00.997089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:00.997102 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:29:00.997254 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:00.997256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:00.997294 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:00.997370 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:00.998215 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:00.998815 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:00.999856 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.000124 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.000246 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.001840 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:01.001844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:01.001846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:01.001880 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:01.003665 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:01.004261 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:01.004265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:01.004269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:01.004369 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:01.004685 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:01.004689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:01.004691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:01.004912 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:01.005085 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:01.005341 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 22:29:01.005476 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:29:01.008908 10062 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:29:01.012688 10062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:29:01.023532 10198 tablet_service.cc:1467] Processing CreateTablet for tablet dc2e89657b8c4cd694dc2037e11a4f71 (DEFAULT_TABLE table=test-table [id=40444c20f60b47278ebe2679481d79b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:01.024606 10198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc2e89657b8c4cd694dc2037e11a4f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:01.027817 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.022148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33755 (local address 127.0.0.1:44574)
0811 22:29:01.022499 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:01.022518 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:01.022634 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:01.023548 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:01.023552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:01.023574 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:01.023697 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:01.023705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.025239 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.025242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:01.026392 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:01.026403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.027314 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.027318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:01.027411 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.027471 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 22:29:01.027563 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:29:01.027653 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":134,"threads_started":1}
I20260811 22:29:01.027990 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.022335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.24.71.195:33755)
0811 22:29:01.022915 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:01.022931 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:01.022976 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:29:01.023075 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:01.023081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:01.023350 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:01.023556 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:01.023808 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.025084 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.026540 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.027159 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.027300 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.027481 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:01.027600 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 22:29:01.027708 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":104,"threads_started":1}
I20260811 22:29:01.029246 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.022574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:45089 (local address 127.0.0.1:38434)
0811 22:29:01.023081 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:01.023097 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:01.023205 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:01.024203 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:01.024206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:01.024226 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:01.024332 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:01.024337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.026072 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.026076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:01.027322 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:01.027336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.028777 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.028781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:01.028885 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.028915 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:29:01.029012 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 22:29:01.029100 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":108,"threads_started":1}
I20260811 22:29:01.029341 10596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.023027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.24.71.194:45089)
0811 22:29:01.023445 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:01.023458 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:01.023483 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:29:01.023561 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:01.023566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:01.024031 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:01.024713 (+ 682us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:01.024894 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.025919 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.027524 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.028611 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.028741 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.028928 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:01.029052 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 22:29:01.029166 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":166,"threads_started":1}
I20260811 22:29:01.029476 10597 tablet_bootstrap.cc:492] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Bootstrap starting.
I20260811 22:29:01.030823 10597 tablet_bootstrap.cc:654] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:01.031582 10501 tablet_service.cc:1467] Processing CreateTablet for tablet dc2e89657b8c4cd694dc2037e11a4f71 (DEFAULT_TABLE table=test-table [id=40444c20f60b47278ebe2679481d79b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:01.032707 10597 tablet_bootstrap.cc:492] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: No bootstrap required, opened a new log
I20260811 22:29:01.032789 10597 ts_tablet_manager.cc:1397] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:29:01.032975 10501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc2e89657b8c4cd694dc2037e11a4f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:01.033166 10597 raft_consensus.cc:348] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:01.033267 10597 raft_consensus.cc:374] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:01.033294 10597 raft_consensus.cc:729] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0c10974f854df19c689ca1b346f608, State: Initialized, Role: FOLLOWER
I20260811 22:29:01.032970 10358 tablet_service.cc:1467] Processing CreateTablet for tablet dc2e89657b8c4cd694dc2037e11a4f71 (DEFAULT_TABLE table=test-table [id=40444c20f60b47278ebe2679481d79b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:01.033614 10597 consensus_queue.cc:260] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:01.034111 10597 ts_tablet_manager.cc:1428] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:29:01.034229 10358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc2e89657b8c4cd694dc2037e11a4f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:01.043345 10602 tablet_bootstrap.cc:492] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: Bootstrap starting.
I20260811 22:29:01.044261 10600 tablet_bootstrap.cc:492] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: Bootstrap starting.
I20260811 22:29:01.046859 10602 tablet_bootstrap.cc:654] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:01.047037 10600 tablet_bootstrap.cc:654] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:01.047987 10602 log.cc:824] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:01.047986 10600 log.cc:824] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:01.050231 10600 tablet_bootstrap.cc:492] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: No bootstrap required, opened a new log
I20260811 22:29:01.050240 10602 tablet_bootstrap.cc:492] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: No bootstrap required, opened a new log
I20260811 22:29:01.050489 10600 ts_tablet_manager.cc:1397] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:29:01.050495 10602 ts_tablet_manager.cc:1397] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:29:01.054279 10600 raft_consensus.cc:348] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:01.054358 10602 raft_consensus.cc:348] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:01.054620 10600 raft_consensus.cc:374] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:01.054615 10602 raft_consensus.cc:374] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:01.054709 10602 raft_consensus.cc:729] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d77b3ecaf514deba0d2ab52f82c8d82, State: Initialized, Role: FOLLOWER
I20260811 22:29:01.054711 10600 raft_consensus.cc:729] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173e05ebd6e1492f96172072b84af1c8, State: Initialized, Role: FOLLOWER
I20260811 22:29:01.055222 10602 consensus_queue.cc:260] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:01.055223 10600 consensus_queue.cc:260] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:01.056404 10583 heartbeater.cc:503] Master 127.24.71.254:40037 was elected leader, sending a full tablet report...
I20260811 22:29:01.056456 10427 heartbeater.cc:503] Master 127.24.71.254:40037 was elected leader, sending a full tablet report...
I20260811 22:29:01.056561 10602 ts_tablet_manager.cc:1428] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:29:01.056643 10600 ts_tablet_manager.cc:1428] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:29:01.085811 10606 raft_consensus.cc:482] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:01.086139 10606 raft_consensus.cc:504] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:01.087530 10606 leader_election.cc:288] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089), 6f0c10974f854df19c689ca1b346f608 (127.24.71.193:36247)
I20260811 22:29:01.092048 10596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.088122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:41195 (local address 127.24.71.194:45089)
0811 22:29:01.088265 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:01.088273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:01.088645 (+ 372us) server_negotiation.cc:407] Connection header received
0811 22:29:01.088831 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:01.088857 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:01.088917 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:01.089048 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:01.089404 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.089892 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.091020 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.091434 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.091569 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.091625 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:01.091850 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:29:01.091924 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:29:01.092063 10609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.087916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:45089 (local address 127.24.71.195:41195)
0811 22:29:01.088463 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:01.088523 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:01.088683 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:01.089059 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:01.089062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:01.089080 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:01.089235 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:01.089244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.090016 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.090020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:01.090889 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:01.090899 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.091568 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.091572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:01.091671 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.091695 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:29:01.091795 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:29:01.091893 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":168,"thread_start_us":118,"threads_started":1}
I20260811 22:29:01.092239 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.088016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:58131 (local address 127.24.71.193:36247)
0811 22:29:01.088420 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:01.088435 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:01.088456 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:29:01.088508 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:01.088511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:01.088565 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:01.088731 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:01.089153 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.089838 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.091055 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.091577 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.091729 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.091969 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:01.092013 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 22:29:01.092095 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":181,"threads_started":1}
I20260811 22:29:01.092237 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.087916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:36247 (local address 127.24.71.195:58131)
0811 22:29:01.088051 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:01.088071 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:01.088177 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:01.088768 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:01.088771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:01.088794 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:01.088980 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:01.088993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.089984 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.089992 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:01.090899 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:01.090911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.091720 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.091724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:01.091863 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.091889 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:29:01.091964 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:29:01.092050 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":29}
I20260811 22:29:01.092792 10379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc2e89657b8c4cd694dc2037e11a4f71" candidate_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173e05ebd6e1492f96172072b84af1c8" is_pre_election: true
I20260811 22:29:01.092882 10222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc2e89657b8c4cd694dc2037e11a4f71" candidate_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0c10974f854df19c689ca1b346f608" is_pre_election: true
I20260811 22:29:01.093184 10379 raft_consensus.cc:2454] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d77b3ecaf514deba0d2ab52f82c8d82 in term 0.
I20260811 22:29:01.093217 10222 raft_consensus.cc:2454] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d77b3ecaf514deba0d2ab52f82c8d82 in term 0.
I20260811 22:29:01.093740 10453 leader_election.cc:302] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173e05ebd6e1492f96172072b84af1c8, 1d77b3ecaf514deba0d2ab52f82c8d82; no voters:
I20260811 22:29:01.094101 10606 raft_consensus.cc:2781] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:29:01.094198 10606 raft_consensus.cc:482] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:29:01.094244 10606 raft_consensus.cc:3037] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:01.095350 10606 raft_consensus.cc:504] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:01.095753 10606 leader_election.cc:288] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [CANDIDATE]: Term 1 election: Requested vote from peers 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089), 6f0c10974f854df19c689ca1b346f608 (127.24.71.193:36247)
I20260811 22:29:01.096184 10379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc2e89657b8c4cd694dc2037e11a4f71" candidate_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173e05ebd6e1492f96172072b84af1c8"
I20260811 22:29:01.096246 10222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc2e89657b8c4cd694dc2037e11a4f71" candidate_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0c10974f854df19c689ca1b346f608"
I20260811 22:29:01.096313 10379 raft_consensus.cc:3037] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:01.096527 10222 raft_consensus.cc:3037] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:01.097496 10379 raft_consensus.cc:2454] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d77b3ecaf514deba0d2ab52f82c8d82 in term 1.
I20260811 22:29:01.097726 10222 raft_consensus.cc:2454] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d77b3ecaf514deba0d2ab52f82c8d82 in term 1.
I20260811 22:29:01.097879 10453 leader_election.cc:302] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173e05ebd6e1492f96172072b84af1c8, 1d77b3ecaf514deba0d2ab52f82c8d82; no voters:
I20260811 22:29:01.098079 10606 raft_consensus.cc:2781] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:01.098390 10606 raft_consensus.cc:686] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 1 LEADER]: Becoming Leader. State: Replica: 1d77b3ecaf514deba0d2ab52f82c8d82, State: Running, Role: LEADER
I20260811 22:29:01.098817 10606 consensus_queue.cc:237] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:01.102461 10061 catalog_manager.cc:5495] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195). New cstate: current_term: 1 leader_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } health_report { overall_health: UNKNOWN } } }
I20260811 22:29:01.119670 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.115397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.24.71.195:33755)
0811 22:29:01.115538 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:01.115543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:01.115559 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:29:01.115653 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:01.115655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:01.115713 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:01.115807 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:29:01.116207 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.116746 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.117760 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:01.118004 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:01.118093 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:01.118484 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:29:01.118942 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:29:01.119065 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:01.119269 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 22:29:01.119442 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:29:01.126753 10379 raft_consensus.cc:1261] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [term 1 FOLLOWER]: Refusing update from remote peer 1d77b3ecaf514deba0d2ab52f82c8d82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:01.126753 10222 raft_consensus.cc:1261] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Refusing update from remote peer 1d77b3ecaf514deba0d2ab52f82c8d82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:01.127547 10610 consensus_queue.cc:1035] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:01.127802 10606 consensus_queue.cc:1035] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:01.135432 10615 mvcc.cc:205] Tried to move back new op lower bound from 7317452149247148032 to 7317452149144010752. Current Snapshot: MvccSnapshot[applied={T|T < 7317452149247148032}]
I20260811 22:29:01.136503 10616 mvcc.cc:205] Tried to move back new op lower bound from 7317452149247148032 to 7317452149144010752. Current Snapshot: MvccSnapshot[applied={T|T < 7317452149247148032}]
I20260811 22:29:01.148842 10061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:01.149068 10061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:01.152055 10061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Sending DeleteTablet for 3 replicas of tablet dc2e89657b8c4cd694dc2037e11a4f71
I20260811 22:29:01.153057 10198 tablet_service.cc:1514] Processing DeleteTablet for tablet dc2e89657b8c4cd694dc2037e11a4f71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:01 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:58600
I20260811 22:29:01.153142 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 10103
I20260811 22:29:01.153227 10358 tablet_service.cc:1514] Processing DeleteTablet for tablet dc2e89657b8c4cd694dc2037e11a4f71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:01 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:38434
I20260811 22:29:01.153849 10501 tablet_service.cc:1514] Processing DeleteTablet for tablet dc2e89657b8c4cd694dc2037e11a4f71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:01 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:44574
I20260811 22:29:01.156944 10627 tablet_replica.cc:330] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: stopping tablet replica
I20260811 22:29:01.157347 10627 raft_consensus.cc:2229] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:29:01.157822 10627 raft_consensus.cc:2258] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:01.158335 10626 tablet_replica.cc:330] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: stopping tablet replica
I20260811 22:29:01.158658 10626 raft_consensus.cc:2229] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:01.158998 10626 raft_consensus.cc:2258] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:01.160918 10626 ts_tablet_manager.cc:1905] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:01.160918 10627 ts_tablet_manager.cc:1905] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:29:01.163812 10045 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:36247 (error 108)
I20260811 22:29:01.163983 10626 ts_tablet_manager.cc:1918] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:01.163983 10627 ts_tablet_manager.cc:1918] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:01.164088 10627 log.cc:1196] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/dc2e89657b8c4cd694dc2037e11a4f71
I20260811 22:29:01.164088 10626 log.cc:1196] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/dc2e89657b8c4cd694dc2037e11a4f71
I20260811 22:29:01.164455 10627 ts_tablet_manager.cc:1939] T dc2e89657b8c4cd694dc2037e11a4f71 P 1d77b3ecaf514deba0d2ab52f82c8d82: Deleting consensus metadata
I20260811 22:29:01.164455 10626 ts_tablet_manager.cc:1939] T dc2e89657b8c4cd694dc2037e11a4f71 P 173e05ebd6e1492f96172072b84af1c8: Deleting consensus metadata
I20260811 22:29:01.165315 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 10278
I20260811 22:29:01.168685 10046 catalog_manager.cc:4841] TS 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755): tablet dc2e89657b8c4cd694dc2037e11a4f71 (table test-table [id=40444c20f60b47278ebe2679481d79b2]) successfully deleted
I20260811 22:29:01.176045 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.175643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:36247 (local address 127.0.0.1:58616)
0811 22:29:01.175813 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:01.175946 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:36247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:29:01.176347 10045 catalog_manager.cc:4584] TS 6f0c10974f854df19c689ca1b346f608 (127.24.71.193:36247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc2e89657b8c4cd694dc2037e11a4f71: Network error: Client connection negotiation failed: client connection to 127.24.71.193:36247: connect: Connection refused (error 111)
I20260811 22:29:01.182240 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 10431
I20260811 22:29:01.184330 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:01.182589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:45089 (local address 127.0.0.1:38438)
0811 22:29:01.184119 (+ 1530us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:01.184201 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.194:45089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1402}
I20260811 22:29:01.191246 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 10027
I20260811 22:29:21.203296 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40037
--webserver_interface=127.24.71.254
--webserver_port=43799
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:40037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:21.324116 10630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:21.324412 10630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:21.324455 10630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:21.328657 10630 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:29:21.328727 10630 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:29:21.328743 10630 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:29:21.328758 10630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:21.328776 10630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:21.328796 10630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:21.328811 10630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:21.328827 10630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:21.334435 10630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:40037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=43799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:21.335531 10630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:21.336637 10630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:29:21.346671 10630 server_base.cc:1034] running on GCE node
W20260811 22:29:21.346680 10635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:21.346855 10639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:21.346680 10636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:21.347587 10630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:21.348894 10630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:21.350090 10630 hybrid_clock.cc:648] HybridClock initialized: now 1786487361350070 us; error 34 us; skew 500 ppm
Aug 11 22:29:21 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:21.357824 10630 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:29:21.358878 10630 webserver.cc:460] Webserver started at http://127.24.71.254:43799/ using document root <none> and password file <none>
I20260811 22:29:21.359455 10630 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:21.359521 10630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:21.363807 10630 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:29:21.366343 10648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:21.368220 10630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:21.368371 10630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "e74eda5596af4414985e0be7f881a01b"
format_stamp: "Formatted at 2026-08-11 22:28:38 on dist-test-slave-mvvj"
server_key: "5ac51a9ed55c7e7533400cadf756dc98"
server_key_iv: "3551baaee4a9537517c12c83540ae0d9"
server_key_version: "encryptionkey@0"
I20260811 22:29:21.369119 10630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:21.385809 10630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:21.389701 10630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:21.389948 10630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:21.398228 10630 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:40037
I20260811 22:29:21.398583 10705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:40037 every 8 connection(s)
I20260811 22:29:21.399519 10630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:21.402453 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 10630
I20260811 22:29:21.402952 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:36247
--local_ip_for_outbound_sockets=127.24.71.193
--tserver_master_addrs=127.24.71.254:40037
--webserver_port=42795
--webserver_interface=127.24.71.193
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:29:21.404359 10706 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:29:21.406831 10706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Bootstrap starting.
I20260811 22:29:21.424921 10706 log.cc:824] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:21.437001 10706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:29:21.437532 10706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Bootstrap complete.
I20260811 22:29:21.442096 10706 raft_consensus.cc:348] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:29:21.442507 10706 raft_consensus.cc:729] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74eda5596af4414985e0be7f881a01b, State: Initialized, Role: FOLLOWER
I20260811 22:29:21.443033 10706 consensus_queue.cc:260] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:29:21.443151 10706 raft_consensus.cc:388] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:21.443238 10706 raft_consensus.cc:482] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:21.443341 10706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:29:21.445334 10706 raft_consensus.cc:504] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:29:21.445688 10706 leader_election.cc:302] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e74eda5596af4414985e0be7f881a01b; no voters:
I20260811 22:29:21.446064 10706 leader_election.cc:288] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:29:21.446169 10711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:29:21.446406 10711 raft_consensus.cc:686] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [term 3 LEADER]: Becoming Leader. State: Replica: e74eda5596af4414985e0be7f881a01b, State: Running, Role: LEADER
I20260811 22:29:21.446769 10711 consensus_queue.cc:237] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } }
I20260811 22:29:21.447355 10706 sys_catalog.cc:564] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:21.447728 10711 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e74eda5596af4414985e0be7f881a01b. Latest consensus state: current_term: 3 leader_uuid: "e74eda5596af4414985e0be7f881a01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } } }
I20260811 22:29:21.447971 10711 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:21.448253 10712 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e74eda5596af4414985e0be7f881a01b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74eda5596af4414985e0be7f881a01b" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40037 } } }
I20260811 22:29:21.448446 10717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:21.448464 10712 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:21.453981 10717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:21.454805 10717 catalog_manager.cc:1259] Loaded cluster ID: 482a836230e04775ac48ff3406c2da6b
I20260811 22:29:21.454875 10717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:21.457461 10717 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:21.458252 10717 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Loaded TSK: 0
I20260811 22:29:21.458350 10717 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e74eda5596af4414985e0be7f881a01b: Loaded TSK: 1
F20260811 22:29:21.459164 10717 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9c7ce009ad google::LogMessage::Fail() at ??:0
@ 0x7f9c7ce0388a google::LogMessage::SendToLog() at ??:0
@ 0x7f9c7ce004de google::LogMessage::Flush() at ??:0
@ 0x7f9c7ce01a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9c7fac418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9c7fabcf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f9c877657bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9c8773f99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9c87740178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f9c87781df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f9c877a082b std::function<>::operator()() at ??:0
@ 0x7f9c8773fb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9c87740f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9c8773d579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9c87781640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9c7f0523d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9c7f052c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f9c7f0543f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9c7f043512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9c806876db start_thread at ??:0
@ 0x7f9c7b8fe61f clone at ??:0
W20260811 22:29:21.565191 10708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:21.565523 10708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:21.565629 10708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:21.570266 10708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:21.570370 10708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:21.570465 10708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:29:21.576236 10708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:36247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=42795
--tserver_master_addrs=127.24.71.254:40037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:21.577445 10708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:21.578563 10708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:21.589073 10765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:21.589149 10766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:21.589417 10768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:21.589426 10708 server_base.cc:1034] running on GCE node
I20260811 22:29:21.590020 10708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:21.590931 10708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:21.592139 10708 hybrid_clock.cc:648] HybridClock initialized: now 1786487361592099 us; error 47 us; skew 500 ppm
Aug 11 22:29:21 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:21.595773 10708 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:29:21.597054 10708 webserver.cc:460] Webserver started at http://127.24.71.193:42795/ using document root <none> and password file <none>
I20260811 22:29:21.597824 10708 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:21.597931 10708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:21.603190 10708 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:29:21.605953 10775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:21.607563 10708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:29:21.607707 10708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "6f0c10974f854df19c689ca1b346f608"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "60faee0731f5dc380dcfd09aeff93ee1"
server_key_iv: "c6960ff9d2f7bff60ba16da17f902479"
server_key_version: "encryptionkey@0"
I20260811 22:29:21.608484 10708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:21.651059 10708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:21.655257 10708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:21.655582 10708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:21.656420 10708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:21.657819 10782 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:29:21.662346 10708 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:29:21.662447 10708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:29:21.662520 10708 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:29:21.664942 10708 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:29:21.665009 10708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260811 22:29:21.665390 10782 tablet_bootstrap.cc:492] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Bootstrap starting.
I20260811 22:29:21.676509 10708 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:36247
I20260811 22:29:21.676856 10906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:36247 every 8 connection(s)
I20260811 22:29:21.677919 10708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:29:21.683650 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 10708
I20260811 22:29:21.684226 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:45089
--local_ip_for_outbound_sockets=127.24.71.194
--tserver_master_addrs=127.24.71.254:40037
--webserver_port=42093
--webserver_interface=127.24.71.194
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:29:21.687223 10782 log.cc:824] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:21.697070 10782 tablet_bootstrap.cc:492] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:29:21.697574 10782 tablet_bootstrap.cc:492] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Bootstrap complete.
I20260811 22:29:21.699105 10782 ts_tablet_manager.cc:1397] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Time spent bootstrapping tablet: real 0.034s user 0.026s sys 0.005s
I20260811 22:29:21.702790 10782 raft_consensus.cc:348] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:21.703420 10782 raft_consensus.cc:729] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0c10974f854df19c689ca1b346f608, State: Initialized, Role: FOLLOWER
I20260811 22:29:21.704051 10782 consensus_queue.cc:260] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:21.705325 10782 ts_tablet_manager.cc:1428] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608: Time spent starting tablet: real 0.006s user 0.009s sys 0.000s
W20260811 22:29:21.756623 10909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:29:21.680583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:51755)
0811 22:29:21.681243 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:21.681323 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:21.682588 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:21.755564 (+ 72976us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":165,"threads_started":1}
W20260811 22:29:21.757790 10907 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:40037 (0 consecutive failures): Network error: Failed to ping master at 127.24.71.254:40037: Client connection negotiation failed: client connection to 127.24.71.254:40037: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:29:21.758806 10909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:21.758449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:59815)
0811 22:29:21.758644 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:21.758734 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 22:29:21.759639 10909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:21.759402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:53701)
0811 22:29:21.759538 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:21.759581 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 22:29:21.759951 10907 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:29:21.820466 10910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:21.820771 10910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:21.820855 10910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:21.824970 10910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:21.825060 10910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:21.825148 10910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:29:21.830744 10910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:45089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=42093
--tserver_master_addrs=127.24.71.254:40037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:21.832011 10910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:21.833088 10910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:21.843618 10918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:21.843649 10919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:21.843742 10910 server_base.cc:1034] running on GCE node
W20260811 22:29:21.843618 10921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:21.844494 10910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:21.845157 10910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:21.846335 10910 hybrid_clock.cc:648] HybridClock initialized: now 1786487361846311 us; error 29 us; skew 500 ppm
Aug 11 22:29:21 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:21.849445 10910 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:29:21.850426 10910 webserver.cc:460] Webserver started at http://127.24.71.194:42093/ using document root <none> and password file <none>
I20260811 22:29:21.851048 10910 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:21.851151 10910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:21.855235 10910 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:29:21.857530 10929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:21.858961 10910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:29:21.859127 10910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "173e05ebd6e1492f96172072b84af1c8"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "de131200ac9f83254134d59fa07e91b4"
server_key_iv: "b5408fa9c2ca16e8b7f5f717184d727b"
server_key_version: "encryptionkey@0"
I20260811 22:29:21.859745 10910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:21.871769 10910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:21.875078 10910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:21.875311 10910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:21.876039 10910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:21.876998 10910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:21.877072 10910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:21.877149 10910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:21.877192 10910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:21.887190 10910 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:45089
I20260811 22:29:21.887217 11057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:45089 every 8 connection(s)
I20260811 22:29:21.888409 10910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:29:21.892769 11060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:21.890847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:44253)
0811 22:29:21.891447 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:21.891972 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":111,"threads_started":1}
W20260811 22:29:21.893496 11058 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:40037 (0 consecutive failures): Network error: Failed to ping master at 127.24.71.254:40037: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
I20260811 22:29:21.893559 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 10910
I20260811 22:29:21.894032 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:33755
--local_ip_for_outbound_sockets=127.24.71.195
--tserver_master_addrs=127.24.71.254:40037
--webserver_port=35051
--webserver_interface=127.24.71.195
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:29:21.894345 11060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:21.894035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:60349)
0811 22:29:21.894189 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:21.894256 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:29:21.895201 11060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:21.894954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:33411)
0811 22:29:21.895095 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:21.895146 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:29:21.895408 11058 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:29:22.012279 11061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:22.012621 11061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:22.012703 11061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:22.017385 11061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:22.017509 11061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:22.017616 11061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:29:22.023373 11061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:33755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=35051
--tserver_master_addrs=127.24.71.254:40037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:22.024689 11061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:22.025870 11061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:22.036701 11067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:22.036789 11068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:22.036715 11070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:22.037132 11061 server_base.cc:1034] running on GCE node
I20260811 22:29:22.037742 11061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:22.038636 11061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:22.039925 11061 hybrid_clock.cc:648] HybridClock initialized: now 1786487362039796 us; error 57 us; skew 500 ppm
Aug 11 22:29:22 dist-test-slave-mvvj krb5kdc[9402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487362, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:22.043223 11061 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:29:22.044361 11061 webserver.cc:460] Webserver started at http://127.24.71.195:35051/ using document root <none> and password file <none>
I20260811 22:29:22.045007 11061 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:22.045079 11061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:22.049422 11061 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:29:22.052033 11077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:22.053602 11061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:22.053745 11061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "1d77b3ecaf514deba0d2ab52f82c8d82"
format_stamp: "Formatted at 2026-08-11 22:28:39 on dist-test-slave-mvvj"
server_key: "7e50c1682733bcef7f9e50ce8549dc40"
server_key_iv: "0832abed552c1ffc954559ba808bbba3"
server_key_version: "encryptionkey@0"
I20260811 22:29:22.054368 11061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:22.073741 11061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:22.077572 11061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:22.077759 11061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:22.078409 11061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:22.079320 11061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:22.079372 11061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:22.079417 11061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:22.079433 11061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:22.089483 11061 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:33755
I20260811 22:29:22.089509 11206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:33755 every 8 connection(s)
I20260811 22:29:22.090574 11061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:29:22.093159 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 11061
I20260811 22:29:22.094929 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:22.092873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:59095)
0811 22:29:22.093812 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:22.094001 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":151,"threads_started":1}
W20260811 22:29:22.095429 11207 heartbeater.cc:644] Failed to heartbeat to 127.24.71.254:40037 (0 consecutive failures): Network error: Failed to ping master at 127.24.71.254:40037: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
I20260811 22:29:22.096205 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:22.095971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:45063)
0811 22:29:22.096092 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:22.096144 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 22:29:22.096938 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:22.096708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:47553)
0811 22:29:22.096818 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:22.096885 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:29:22.097168 11207 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:29:22.761227 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:22.760671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:36031)
0811 22:29:22.761012 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:22.761071 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":134,"threads_started":1}
I20260811 22:29:22.896523 11212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:22.896076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:57397)
0811 22:29:22.896352 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:22.896410 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":87,"threads_started":1}
I20260811 22:29:23.098162 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:23.097749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:49013)
0811 22:29:23.097996 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:23.098052 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260811 22:29:23.158164 11214 raft_consensus.cc:482] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:23.158576 11214 raft_consensus.cc:504] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:23.159806 11214 leader_election.cc:288] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755), 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089)
Aug 11 22:29:23 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.194@KRBTEST.COM
Aug 11 22:29:23 dist-test-slave-mvvj krb5kdc[9402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.195@KRBTEST.COM
I20260811 22:29:23.174491 11215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:23.160294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:51845 (local address 127.24.71.195:33755)
0811 22:29:23.160711 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:23.160735 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:23.160774 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:29:23.161668 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:23.161687 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:23.162107 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:23.162419 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:23.163764 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:23.164856 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:23.166171 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:23.166641 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:23.166817 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:23.169915 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:23.169925 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:23.169936 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:23.169965 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:23.172039 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:23.172572 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:23.172576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:23.172581 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:23.172754 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:23.173061 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:23.173064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:23.173066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:23.173536 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:23.173672 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:23.174049 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 22:29:23.174312 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":104,"threads_started":1}
I20260811 22:29:23.174491 11216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:23.160539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:45089 (local address 127.24.71.193:37805)
0811 22:29:23.160940 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:23.160955 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:23.161069 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:23.162609 (+ 1540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:23.162612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:23.163082 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:23.163580 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:23.163596 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:23.165068 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:23.165071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:23.166038 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:23.166048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:23.166769 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:23.166773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:23.167008 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:23.167673 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:23.167689 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:23.169345 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:23.171690 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:23.171698 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:23.171714 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:23.172106 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:23.172514 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:23.172517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:23.172519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:23.172702 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:23.173383 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:23.173389 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:23.173665 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 22:29:23.173895 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 22:29:23.174275 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":150,"spinlock_wait_cycles":13312,"thread_start_us":94,"threads_started":1}
I20260811 22:29:23.174538 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:23.160157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33755 (local address 127.24.71.193:51845)
0811 22:29:23.160313 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:23.160328 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:23.160428 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:23.162483 (+ 2055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:23.162490 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:23.163068 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:23.163580 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:23.163596 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:23.165069 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:23.165072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:23.166035 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:23.166045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:23.166772 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:23.166775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:23.167006 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:23.167744 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:23.167757 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:23.169730 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:23.172174 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:23.172179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:23.172184 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:23.172431 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:23.172869 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:23.172872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:23.172874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:23.172960 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:23.173653 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:23.173655 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:23.173736 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 22:29:23.173912 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 22:29:23.174365 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":268,"spinlock_wait_cycles":7808}
I20260811 22:29:23.174552 11217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:23.160755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:37805 (local address 127.24.71.194:45089)
0811 22:29:23.161227 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:23.161259 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:23.161312 (+ 53us) server_negotiation.cc:407] Connection header received
0811 22:29:23.162100 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:23.162106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:23.162389 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:23.162663 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:23.163752 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:23.164868 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:23.166167 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:23.166637 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:23.166817 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:23.169619 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:23.169627 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:23.169648 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:23.169686 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:23.171550 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:23.172241 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:23.172247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:23.172251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:23.172371 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:23.172826 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:23.172829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:23.172830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:23.173249 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:23.173398 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:23.174023 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 22:29:23.174349 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":108,"threads_started":1}
W20260811 22:29:23.177377 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
W20260811 22:29:23.177666 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
I20260811 22:29:23.177762 10778 leader_election.cc:302] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f0c10974f854df19c689ca1b346f608; no voters: 173e05ebd6e1492f96172072b84af1c8, 1d77b3ecaf514deba0d2ab52f82c8d82
I20260811 22:29:23.178007 11214 raft_consensus.cc:2726] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:29:23.762442 11218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:23.762016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:33469)
0811 22:29:23.762279 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:23.762333 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260811 22:29:23.897877 11219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:23.897434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:38867)
0811 22:29:23.897714 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:23.897776 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260811 22:29:24.099692 11220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:24.099202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:51055)
0811 22:29:24.099520 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:24.099588 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":86,"threads_started":1}
I20260811 22:29:24.763770 11221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:24.763254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:42231)
0811 22:29:24.763579 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:24.763668 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
I20260811 22:29:24.899317 11222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:24.898806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:50785)
0811 22:29:24.899138 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:24.899202 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":99,"threads_started":1}
I20260811 22:29:24.952168 11223 raft_consensus.cc:482] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:24.952310 11223 raft_consensus.cc:504] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:24.952696 11223 leader_election.cc:288] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755), 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089)
W20260811 22:29:24.953698 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
W20260811 22:29:24.953889 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
I20260811 22:29:24.953953 10778 leader_election.cc:302] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f0c10974f854df19c689ca1b346f608; no voters: 173e05ebd6e1492f96172072b84af1c8, 1d77b3ecaf514deba0d2ab52f82c8d82
I20260811 22:29:24.954188 11223 raft_consensus.cc:2726] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:29:25.101162 11225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:25.100648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:38227)
0811 22:29:25.100962 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:25.101047 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":106,"threads_started":1}
I20260811 22:29:25.765223 11226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:25.764771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:46823)
0811 22:29:25.765060 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:25.765123 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
I20260811 22:29:25.900696 11227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:25.900210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:46533)
0811 22:29:25.900531 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:25.900597 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":81,"threads_started":1}
I20260811 22:29:26.102626 11228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:26.102082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:49211)
0811 22:29:26.102451 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:26.102520 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":89,"threads_started":1}
I20260811 22:29:26.766822 11229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:26.766341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:41807)
0811 22:29:26.766663 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:26.766724 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":88,"threads_started":1}
I20260811 22:29:26.902103 11231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:26.901698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:54209)
0811 22:29:26.901947 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:26.902004 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260811 22:29:27.104179 11232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:27.103644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:35105)
0811 22:29:27.104015 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:27.104077 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":104,"threads_started":1}
I20260811 22:29:27.768347 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:27.767799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:45843)
0811 22:29:27.768161 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:27.768229 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":80,"threads_started":1}
I20260811 22:29:27.903496 11237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:27.902994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:37415)
0811 22:29:27.903291 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:27.903355 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
I20260811 22:29:27.932547 11238 raft_consensus.cc:482] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:27.932730 11238 raft_consensus.cc:504] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:27.933208 11238 leader_election.cc:288] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755), 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089)
W20260811 22:29:27.934124 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
W20260811 22:29:27.934283 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
I20260811 22:29:27.934347 10778 leader_election.cc:302] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f0c10974f854df19c689ca1b346f608; no voters: 173e05ebd6e1492f96172072b84af1c8, 1d77b3ecaf514deba0d2ab52f82c8d82
I20260811 22:29:27.934543 11238 raft_consensus.cc:2726] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:29:28.105576 11239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:28.105077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:33547)
0811 22:29:28.105423 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:28.105480 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":96,"threads_started":1}
I20260811 22:29:28.769726 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:28.769209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:48139)
0811 22:29:28.769548 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:28.769622 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":81,"threads_started":1}
I20260811 22:29:28.904934 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:28.904383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:42909)
0811 22:29:28.904739 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:28.904804 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":141,"threads_started":1}
I20260811 22:29:29.106892 11243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:29.106402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:37613)
0811 22:29:29.106732 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:29.106788 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":96,"threads_started":1}
I20260811 22:29:29.771384 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:29.770850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:39173)
0811 22:29:29.771212 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:29.771281 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":87,"threads_started":1}
I20260811 22:29:29.906200 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:29.905756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:60299)
0811 22:29:29.906037 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:29.906099 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
I20260811 22:29:30.108274 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:30.107731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:51139)
0811 22:29:30.108096 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:30.108171 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":146,"threads_started":1}
I20260811 22:29:30.772856 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:30.772366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:39469)
0811 22:29:30.772687 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:30.772754 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":81,"threads_started":1}
I20260811 22:29:30.907644 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:30.907159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:59129)
0811 22:29:30.907473 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:30.907542 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":82,"threads_started":1}
I20260811 22:29:31.109899 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:31.109305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:49109)
0811 22:29:31.109694 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:31.109782 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":135,"threads_started":1}
I20260811 22:29:31.145036 11250 raft_consensus.cc:482] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:31.145182 11250 raft_consensus.cc:504] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:31.145623 11250 leader_election.cc:288] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755), 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089)
W20260811 22:29:31.146562 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
W20260811 22:29:31.146718 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
I20260811 22:29:31.146778 10778 leader_election.cc:302] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f0c10974f854df19c689ca1b346f608; no voters: 173e05ebd6e1492f96172072b84af1c8, 1d77b3ecaf514deba0d2ab52f82c8d82
I20260811 22:29:31.146970 11250 raft_consensus.cc:2726] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:29:31.774601 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:31.774056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:55295)
0811 22:29:31.774420 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:31.774491 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":123,"threads_started":1}
I20260811 22:29:31.909279 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:31.908772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:55067)
0811 22:29:31.909100 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:31.909172 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":90,"threads_started":1}
I20260811 22:29:32.111341 11253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:32.110847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:36951)
0811 22:29:32.111166 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:32.111232 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":97,"threads_started":1}
I20260811 22:29:32.775986 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:32.775479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:41555)
0811 22:29:32.775793 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:32.775882 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":81,"threads_started":1}
I20260811 22:29:32.910771 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:32.910218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:42393)
0811 22:29:32.910604 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:32.910670 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":135,"threads_started":1}
I20260811 22:29:33.112980 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:33.112452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:57283)
0811 22:29:33.112784 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:33.112856 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":87,"threads_started":1}
I20260811 22:29:33.777494 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:33.777021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:60791)
0811 22:29:33.777325 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:33.777393 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":97,"threads_started":1}
I20260811 22:29:33.912353 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:33.911793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:34731)
0811 22:29:33.912175 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:33.912248 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":105,"threads_started":1}
I20260811 22:29:34.114625 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:34.114114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:42841)
0811 22:29:34.114415 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:34.114490 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":114,"threads_started":1}
I20260811 22:29:34.778908 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:34.778452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:42965)
0811 22:29:34.778746 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:34.778807 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
I20260811 22:29:34.913981 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:34.913484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:35097)
0811 22:29:34.913799 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:34.913860 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":94,"threads_started":1}
I20260811 22:29:35.116106 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:35.115503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:53975)
0811 22:29:35.115931 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:35.116001 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":82,"threads_started":1}
I20260811 22:29:35.521360 11263 raft_consensus.cc:482] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:35.521505 11263 raft_consensus.cc:504] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d77b3ecaf514deba0d2ab52f82c8d82" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33755 } } peers { permanent_uuid: "173e05ebd6e1492f96172072b84af1c8" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 45089 } } peers { permanent_uuid: "6f0c10974f854df19c689ca1b346f608" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36247 } }
I20260811 22:29:35.521929 11263 leader_election.cc:288] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755), 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089)
W20260811 22:29:35.522997 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173e05ebd6e1492f96172072b84af1c8 (127.24.71.194:45089): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
W20260811 22:29:35.523175 10778 leader_election.cc:341] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d77b3ecaf514deba0d2ab52f82c8d82 (127.24.71.195:33755): Not found: Tablet not found: dc2e89657b8c4cd694dc2037e11a4f71
I20260811 22:29:35.523248 10778 leader_election.cc:302] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f0c10974f854df19c689ca1b346f608; no voters: 173e05ebd6e1492f96172072b84af1c8, 1d77b3ecaf514deba0d2ab52f82c8d82
I20260811 22:29:35.523464 11263 raft_consensus.cc:2726] T dc2e89657b8c4cd694dc2037e11a4f71 P 6f0c10974f854df19c689ca1b346f608 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:29:35.780417 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:35.779937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:34567)
0811 22:29:35.780242 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:35.780310 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":83,"threads_started":1}
I20260811 22:29:35.915601 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:35.915108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:53151)
0811 22:29:35.915420 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:35.915498 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":118,"threads_started":1}
I20260811 22:29:36.117753 11266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:36.117150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:53039)
0811 22:29:36.117506 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:36.117598 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":121,"threads_started":1}
I20260811 22:29:36.782130 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:36.781684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.193:52093)
0811 22:29:36.781950 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:36.782013 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":84,"threads_started":1}
I20260811 22:29:36.917271 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:36.916675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.194:58115)
0811 22:29:36.917090 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:36.917162 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":85,"threads_started":1}
I20260811 22:29:37.101624 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 10708
I20260811 22:29:37.111312 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 10910
I20260811 22:29:37.119503 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:37.118983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40037 (local address 127.24.71.195:37637)
0811 22:29:37.119317 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:37.119387 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.254:40037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":101,"threads_started":1}
I20260811 22:29:37.121100 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 11061
2026-08-11T22:29:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59704 ms)
[----------] 30 tests from SecurityITest (256010 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:37 dist-test-slave-mvvj krb5kdc[11272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:37 dist-test-slave-mvvj krb5kdc[11272](info): set up 2 sockets
Aug 11 22:29:37 dist-test-slave-mvvj krb5kdc[11272](info): commencing operation
krb5kdc: starting...
W20260811 22:29:38.289422 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:38 dist-test-slave-mvvj krb5kdc[11272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:29:38.437271 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40533
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:40533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:38.556120 11287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:38.556457 11287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:38.556542 11287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:38.560688 11287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:38.560787 11287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:38.560827 11287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:38.560887 11287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:38.560925 11287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:38.566431 11287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:40533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:40533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:38.567629 11287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:38.568809 11287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:29:38.578115 11287 server_base.cc:1034] running on GCE node
W20260811 22:29:38.578332 11294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:38.578449 11296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:38.578191 11293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:38.579233 11287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:38.580457 11287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:38.581650 11287 hybrid_clock.cc:648] HybridClock initialized: now 1786487378581615 us; error 49 us; skew 500 ppm
Aug 11 22:29:38 dist-test-slave-mvvj krb5kdc[11272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487378, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:38.584789 11287 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:29:38.585755 11287 webserver.cc:460] Webserver started at http://127.24.71.254:43575/ using document root <none> and password file <none>
I20260811 22:29:38.586372 11287 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:38.586470 11287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:38.586724 11287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:38.589097 11287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "dbdbb48494da434885584d7574621871"
format_stamp: "Formatted at 2026-08-11 22:29:38 on dist-test-slave-mvvj"
server_key: "95efcef9262ff5cc73165d88d265bff4"
server_key_iv: "47e0efc8f7f15b76a97b86098e0a6fa8"
server_key_version: "encryptionkey@0"
I20260811 22:29:38.589751 11287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "dbdbb48494da434885584d7574621871"
format_stamp: "Formatted at 2026-08-11 22:29:38 on dist-test-slave-mvvj"
server_key: "95efcef9262ff5cc73165d88d265bff4"
server_key_iv: "47e0efc8f7f15b76a97b86098e0a6fa8"
server_key_version: "encryptionkey@0"
I20260811 22:29:38.593930 11287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:29:38.596758 11304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:38.598201 11287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:38.598376 11287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "dbdbb48494da434885584d7574621871"
format_stamp: "Formatted at 2026-08-11 22:29:38 on dist-test-slave-mvvj"
server_key: "95efcef9262ff5cc73165d88d265bff4"
server_key_iv: "47e0efc8f7f15b76a97b86098e0a6fa8"
server_key_version: "encryptionkey@0"
I20260811 22:29:38.598534 11287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:38.626628 11287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:38.630613 11287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:38.630879 11287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:38.638803 11287 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:40533
I20260811 22:29:38.638842 11366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:40533 every 8 connection(s)
I20260811 22:29:38.639863 11287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:38.642874 11367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:38.645735 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 11287
I20260811 22:29:38.645891 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:38.646245 24863 external_mini_cluster.cc:1428] Setting key bfc5e4d30c05dfe6593c77a2f84f95de
I20260811 22:29:38.649955 11367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871: Bootstrap starting.
I20260811 22:29:38.652556 11367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:38.653333 11367 log.cc:824] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:38.655340 11367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871: No bootstrap required, opened a new log
Aug 11 22:29:38 dist-test-slave-mvvj krb5kdc[11272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:38.658613 11367 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdbb48494da434885584d7574621871" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40533 } }
I20260811 22:29:38.658883 11367 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:38.658967 11367 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbdbb48494da434885584d7574621871, State: Initialized, Role: FOLLOWER
I20260811 22:29:38.659523 11367 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdbb48494da434885584d7574621871" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40533 } }
I20260811 22:29:38.659693 11367 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:38.659772 11367 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:38.659905 11367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:38.661201 11367 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdbb48494da434885584d7574621871" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40533 } }
I20260811 22:29:38.661582 11367 leader_election.cc:302] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbdbb48494da434885584d7574621871; no voters:
I20260811 22:29:38.661959 11367 leader_election.cc:288] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:38.662070 11372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:38.662266 11372 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [term 1 LEADER]: Becoming Leader. State: Replica: dbdbb48494da434885584d7574621871, State: Running, Role: LEADER
I20260811 22:29:38.662353 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:38.648017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35792 (local address 127.24.71.254:40533)
0811 22:29:38.648552 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:38.648569 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:38.648613 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:29:38.649304 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:38.649323 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:38.649684 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:38.650084 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:38.650964 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:38.652111 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:38.653080 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:38.653636 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:38.653810 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:38.656499 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:38.656513 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:38.656530 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:38.656561 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:38.658920 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:38.659450 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:38.659456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:38.659460 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:38.659581 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:38.659920 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:38.659923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:38.659924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:38.660411 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:38.660589 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:38.660901 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 22:29:38.661227 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":170,"threads_started":1}
I20260811 22:29:38.662655 11372 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdbb48494da434885584d7574621871" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40533 } }
I20260811 22:29:38.663367 11367 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:38.663949 11372 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbdbb48494da434885584d7574621871. Latest consensus state: current_term: 1 leader_uuid: "dbdbb48494da434885584d7574621871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdbb48494da434885584d7574621871" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40533 } } }
I20260811 22:29:38.664151 11372 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:38.664440 11373 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbdbb48494da434885584d7574621871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbdbb48494da434885584d7574621871" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 40533 } } }
I20260811 22:29:38.664551 11373 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:38.665019 11378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:38.668569 11378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:38.674429 11378 catalog_manager.cc:1347] Generated new cluster ID: 4be62c6e72a94b3aa66992a6d0e98b9e
I20260811 22:29:38.674535 11378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:38.687906 11378 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:38.688994 11378 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:38.697407 11378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871: Generated new TSK 0
I20260811 22:29:38.698223 11378 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:29:38.741114 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40533
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:38.892936 11397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:38.893249 11397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:38.893298 11397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:38.897444 11397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:38.897521 11397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:38.897583 11397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:29:38.903261 11397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:38.904531 11397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:38.905594 11397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:38.915992 11402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:38.916028 11405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:38.916056 11403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:38.916375 11397 server_base.cc:1034] running on GCE node
I20260811 22:29:38.916810 11397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:38.917505 11397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:38.918699 11397 hybrid_clock.cc:648] HybridClock initialized: now 1786487378918685 us; error 25 us; skew 500 ppm
Aug 11 22:29:38 dist-test-slave-mvvj krb5kdc[11272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487378, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:38.921988 11397 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:29:38.922966 11397 webserver.cc:460] Webserver started at http://127.24.71.193:39109/ using document root <none> and password file <none>
I20260811 22:29:38.923573 11397 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:38.923651 11397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:38.923887 11397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:38.926097 11397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "02314ef0b92f4b56a46a55370ddd582c"
format_stamp: "Formatted at 2026-08-11 22:29:38 on dist-test-slave-mvvj"
server_key: "1cb0dec71c71acf62d2c94b22256249b"
server_key_iv: "68b4c316ce988d44a50dd7c905c3ae80"
server_key_version: "encryptionkey@0"
I20260811 22:29:38.926689 11397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "02314ef0b92f4b56a46a55370ddd582c"
format_stamp: "Formatted at 2026-08-11 22:29:38 on dist-test-slave-mvvj"
server_key: "1cb0dec71c71acf62d2c94b22256249b"
server_key_iv: "68b4c316ce988d44a50dd7c905c3ae80"
server_key_version: "encryptionkey@0"
I20260811 22:29:38.930853 11397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:29:38.933672 11412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:38.935088 11397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:29:38.935264 11397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "02314ef0b92f4b56a46a55370ddd582c"
format_stamp: "Formatted at 2026-08-11 22:29:38 on dist-test-slave-mvvj"
server_key: "1cb0dec71c71acf62d2c94b22256249b"
server_key_iv: "68b4c316ce988d44a50dd7c905c3ae80"
server_key_version: "encryptionkey@0"
I20260811 22:29:38.935421 11397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:38.953789 11397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:38.957408 11397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:38.957633 11397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:38.958279 11397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:38.959210 11397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:38.959296 11397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:38.959407 11397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:38.959460 11397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:38.969672 11397 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:36539
I20260811 22:29:38.969692 11539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:36539 every 8 connection(s)
I20260811 22:29:38.970742 11397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:29:38.980973 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 11397
I20260811 22:29:38.981132 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
Aug 11 22:29:38 dist-test-slave-mvvj krb5kdc[11272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487378, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:38.981609 24863 external_mini_cluster.cc:1428] Setting key 369af4ed365b86dc0706be98087c0eb1
I20260811 22:29:38.986858 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:38.972923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:48283 (local address 127.24.71.254:40533)
0811 22:29:38.973076 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:38.973080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:38.973996 (+ 916us) server_negotiation.cc:407] Connection header received
0811 22:29:38.975501 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:38.975504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:38.975555 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:38.975635 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:38.976969 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:38.977592 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:38.978786 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:38.979035 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:38.979124 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:38.982306 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:38.982310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:38.982313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:38.982347 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:38.984301 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:38.984904 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:38.984911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:38.984916 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:38.985017 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:38.985361 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:38.985367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:38.985372 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:38.985646 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:38.985761 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:38.986547 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 22:29:38.986699 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:29:38.988202 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:38.973348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40533 (local address 127.24.71.193:48283)
0811 22:29:38.973836 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:38.973881 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:38.975250 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:38.975813 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:38.975820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:38.976254 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:38.976803 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:38.976818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:38.977720 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:38.977722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:38.978658 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:38.978668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:38.979188 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:38.979190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:38.979336 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:38.980101 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:38.980118 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:38.981982 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:38.984460 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:38.984466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:38.984482 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:38.984786 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:38.985117 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:38.985119 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:38.985121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:38.985267 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:38.985775 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:38.985786 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:38.986216 (+ 430us) client_negotiation.cc:769] Sending connection context
0811 22:29:38.986467 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 22:29:38.986897 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":135,"threads_started":1}
I20260811 22:29:38.989796 11540 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40533
I20260811 22:29:38.990103 11540 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:38.990811 11540 heartbeater.cc:511] Master 127.24.71.254:40533 requested a full tablet report, sending...
I20260811 22:29:38.992682 11323 ts_manager.cc:194] Registered new tserver with Master: 02314ef0b92f4b56a46a55370ddd582c (127.24.71.193:36539)
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:29:38.994040 11323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:48283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:29:39.015941 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40533
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:39.136624 11546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:39.136983 11546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:39.137081 11546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:39.141882 11546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:39.142004 11546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:39.142109 11546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:29:39.148059 11546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:39.149294 11546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:39.150429 11546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:39.160506 11551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:39.160563 11552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:39.160578 11554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:39.160979 11546 server_base.cc:1034] running on GCE node
I20260811 22:29:39.161515 11546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:39.162259 11546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:39.163414 11546 hybrid_clock.cc:648] HybridClock initialized: now 1786487379163388 us; error 36 us; skew 500 ppm
Aug 11 22:29:39 dist-test-slave-mvvj krb5kdc[11272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487379, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:39.166474 11546 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:29:39.167421 11546 webserver.cc:460] Webserver started at http://127.24.71.194:41157/ using document root <none> and password file <none>
I20260811 22:29:39.168090 11546 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:39.168157 11546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:39.168438 11546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:39.170660 11546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "88b478e6b8404466a9a60acf7ce55001"
format_stamp: "Formatted at 2026-08-11 22:29:39 on dist-test-slave-mvvj"
server_key: "7115b5b9978ffc8ef767090e36bf1c8f"
server_key_iv: "0bbd3a3f1d3f75bbdf9960a8a80d9add"
server_key_version: "encryptionkey@0"
I20260811 22:29:39.171281 11546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "88b478e6b8404466a9a60acf7ce55001"
format_stamp: "Formatted at 2026-08-11 22:29:39 on dist-test-slave-mvvj"
server_key: "7115b5b9978ffc8ef767090e36bf1c8f"
server_key_iv: "0bbd3a3f1d3f75bbdf9960a8a80d9add"
server_key_version: "encryptionkey@0"
I20260811 22:29:39.175282 11546 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:29:39.178010 11561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:39.179555 11546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:29:39.179697 11546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "88b478e6b8404466a9a60acf7ce55001"
format_stamp: "Formatted at 2026-08-11 22:29:39 on dist-test-slave-mvvj"
server_key: "7115b5b9978ffc8ef767090e36bf1c8f"
server_key_iv: "0bbd3a3f1d3f75bbdf9960a8a80d9add"
server_key_version: "encryptionkey@0"
I20260811 22:29:39.179818 11546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:39.210493 11546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:39.213905 11546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:39.214133 11546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:39.214789 11546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:39.215713 11546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:39.215762 11546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:39.215832 11546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:39.215899 11546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:39.225610 11546 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:38717
I20260811 22:29:39.225636 11693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:38717 every 8 connection(s)
I20260811 22:29:39.226627 11546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:29:39.234757 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 11546
I20260811 22:29:39.234948 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:29:39.235356 24863 external_mini_cluster.cc:1428] Setting key 5b3f9f93bda5d6a4dd4d23241c9536a5
Aug 11 22:29:39 dist-test-slave-mvvj krb5kdc[11272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487379, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:39.243144 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.228750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:59755 (local address 127.24.71.254:40533)
0811 22:29:39.228936 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.228942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.229848 (+ 906us) server_negotiation.cc:407] Connection header received
0811 22:29:39.231191 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:39.231194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:39.231247 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:39.231325 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:39.232682 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.233224 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.234537 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.234922 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.235063 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.238199 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:39.238206 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:39.238212 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:39.238248 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:39.240323 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:39.241085 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:39.241088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:39.241089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:39.241172 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:39.241726 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:39.241729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:39.241730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:39.241962 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:39.242060 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:39.242880 (+ 820us) server_negotiation.cc:299] Negotiation successful
0811 22:29:39.243009 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:29:39.244742 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.229172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40533 (local address 127.24.71.194:59755)
0811 22:29:39.229667 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:39.229723 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:39.230972 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:39.231549 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:39.231556 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:39.232009 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:39.232532 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.232547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.233347 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.233349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.234362 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.234373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.235143 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.235147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.235305 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.236117 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:39.236141 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:39.237992 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:39.240466 (+ 2474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:39.240477 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:39.240501 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:39.240950 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:39.241335 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:39.241340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:39.241344 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:39.241577 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:39.242113 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:39.242124 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:39.242537 (+ 413us) client_negotiation.cc:769] Sending connection context
0811 22:29:39.242790 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 22:29:39.243301 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":128,"threads_started":1}
I20260811 22:29:39.246395 11695 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40533
I20260811 22:29:39.246696 11695 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:39.247283 11695 heartbeater.cc:511] Master 127.24.71.254:40533 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:29:39.248541 11323 ts_manager.cc:194] Registered new tserver with Master: 88b478e6b8404466a9a60acf7ce55001 (127.24.71.194:38717)
I20260811 22:29:39.249589 11323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:59755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:29:39.271767 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40533
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:39.389971 11703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:39.390291 11703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:39.390364 11703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:39.394515 11703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:39.394619 11703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:39.394718 11703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:29:39.400463 11703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:40533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:39.401648 11703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:39.402801 11703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:29:39.412963 11703 server_base.cc:1034] running on GCE node
W20260811 22:29:39.412997 11710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:39.412994 11712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:39.412971 11709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:39.413645 11703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:39.414268 11703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:39.415467 11703 hybrid_clock.cc:648] HybridClock initialized: now 1786487379415469 us; error 47 us; skew 500 ppm
Aug 11 22:29:39 dist-test-slave-mvvj krb5kdc[11272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487379, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:39.418390 11703 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:29:39.419363 11703 webserver.cc:460] Webserver started at http://127.24.71.195:41253/ using document root <none> and password file <none>
I20260811 22:29:39.420063 11703 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:39.420163 11703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:39.420399 11703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:39.422574 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "b7427144fa17406e8e7bce8ed543eee8"
format_stamp: "Formatted at 2026-08-11 22:29:39 on dist-test-slave-mvvj"
server_key: "d1ede2094781fbc2903521d890969f00"
server_key_iv: "ef82cdc679ad53095218d2c9eeeca2eb"
server_key_version: "encryptionkey@0"
I20260811 22:29:39.423168 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "b7427144fa17406e8e7bce8ed543eee8"
format_stamp: "Formatted at 2026-08-11 22:29:39 on dist-test-slave-mvvj"
server_key: "d1ede2094781fbc2903521d890969f00"
server_key_iv: "ef82cdc679ad53095218d2c9eeeca2eb"
server_key_version: "encryptionkey@0"
I20260811 22:29:39.427150 11703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:29:39.429865 11721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:39.431348 11703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:29:39.431555 11703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "b7427144fa17406e8e7bce8ed543eee8"
format_stamp: "Formatted at 2026-08-11 22:29:39 on dist-test-slave-mvvj"
server_key: "d1ede2094781fbc2903521d890969f00"
server_key_iv: "ef82cdc679ad53095218d2c9eeeca2eb"
server_key_version: "encryptionkey@0"
I20260811 22:29:39.431689 11703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:39.473160 11703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:39.476482 11703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:39.476711 11703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:39.477348 11703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:39.478266 11703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:39.478354 11703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:39.478448 11703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:39.478492 11703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:39.488411 11703 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:44215
I20260811 22:29:39.488432 11853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:44215 every 8 connection(s)
I20260811 22:29:39.489418 11703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:29:39.495126 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 11703
I20260811 22:29:39.495312 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:29:39.495622 24863 external_mini_cluster.cc:1428] Setting key fbc7c8236dabd1e8ba1f0bf2babcb52a
Aug 11 22:29:39 dist-test-slave-mvvj krb5kdc[11272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487379, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:39.504232 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.491349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:42421 (local address 127.24.71.254:40533)
0811 22:29:39.491504 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.491508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.492428 (+ 920us) server_negotiation.cc:407] Connection header received
0811 22:29:39.493792 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:39.493796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:39.493857 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:39.493970 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:39.495190 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.495966 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.497431 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.497758 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.497869 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.500473 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:39.500477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:39.500479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:39.500512 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:39.502054 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:39.502577 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:39.502584 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:39.502585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:39.502670 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:39.503057 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:39.503061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:39.503062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:39.503254 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:39.503372 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:39.503915 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 22:29:39.504059 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:29:39.505147 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.491759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:40533 (local address 127.24.71.195:42421)
0811 22:29:39.492276 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:39.492320 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:39.493558 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:39.494166 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:39.494173 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:39.494549 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:39.495035 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.495049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.496134 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.496137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.497312 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.497322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.497933 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.497937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.498083 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.498748 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:39.498764 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:39.500292 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:39.502181 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:39.502185 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:39.502199 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:39.502455 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:39.502762 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:39.502764 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:39.502766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:39.502901 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:39.503369 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:39.503377 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:39.503649 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 22:29:39.503868 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 22:29:39.504211 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":124,"threads_started":1}
I20260811 22:29:39.506346 11854 heartbeater.cc:347] Connected to a master server at 127.24.71.254:40533
I20260811 22:29:39.506611 11854 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:39.507136 11854 heartbeater.cc:511] Master 127.24.71.254:40533 requested a full tablet report, sending...
I20260811 22:29:39.508311 11323 ts_manager.cc:194] Registered new tserver with Master: b7427144fa17406e8e7bce8ed543eee8 (127.24.71.195:44215)
I20260811 22:29:39.509073 11323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:42421
I20260811 22:29:39.509788 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:29:39.520040 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.511999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35796 (local address 127.24.71.254:40533)
0811 22:29:39.512123 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.512129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.512331 (+ 202us) server_negotiation.cc:407] Connection header received
0811 22:29:39.512495 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:39.512499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:39.512545 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:39.512640 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:39.513380 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.513900 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.514774 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.515005 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.515119 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.516598 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:39.516602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:39.516605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:39.516638 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:39.518041 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:39.518629 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:39.518634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:39.518636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:39.518727 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:39.519044 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:39.519047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:39.519049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:39.519291 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:39.519447 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:39.519707 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 22:29:39.519857 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:29:39.523702 11324 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:29:39.527099 11324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:29:39.542995 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.537194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:38717 (local address 127.0.0.1:40620)
0811 22:29:39.537490 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:39.537504 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:39.537592 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:39.538481 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:39.538485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:39.538511 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:39.538628 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.538638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.539738 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.539745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.541526 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.541532 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.542375 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.542379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.542499 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.542542 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:29:39.542657 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:29:39.542818 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":74,"thread_start_us":101,"threads_started":1}
I20260811 22:29:39.543017 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.536837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:36539 (local address 127.0.0.1:35216)
0811 22:29:39.537109 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:39.537121 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:39.537231 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:39.538644 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:39.538647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:39.538665 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:39.538764 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.538771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.540286 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.540291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.541487 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.541496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.542646 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.542652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.542769 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.542790 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:29:39.542862 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:29:39.542921 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":92,"thread_start_us":104,"threads_started":1}
I20260811 22:29:39.543361 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.536462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59496 (local address 127.24.71.195:44215)
0811 22:29:39.536921 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.536934 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.537359 (+ 425us) server_negotiation.cc:407] Connection header received
0811 22:29:39.537597 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:39.537603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:39.537821 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:39.537961 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:39.538523 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.539749 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.541677 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.542475 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.542600 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.542744 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:39.543113 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 22:29:39.543238 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":161,"threads_started":1}
I20260811 22:29:39.543354 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.537741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35216 (local address 127.24.71.193:36539)
0811 22:29:39.538168 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.538175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.538196 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:29:39.538265 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:39.538270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:39.538486 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:39.538913 (+ 427us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:39.539048 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.540135 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.541615 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.542475 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.542633 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.542750 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:39.542920 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:29:39.543037 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":134,"threads_started":1}
I20260811 22:29:39.543773 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.537381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40620 (local address 127.24.71.194:38717)
0811 22:29:39.538082 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.538097 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.538119 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:29:39.538196 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:39.538205 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:39.538351 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:39.538502 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:39.538741 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.539610 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.541672 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.542247 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.542376 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.542547 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:39.543522 (+ 975us) server_negotiation.cc:299] Negotiation successful
0811 22:29:39.543642 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":202,"threads_started":1}
I20260811 22:29:39.543947 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.536348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:44215 (local address 127.0.0.1:59496)
0811 22:29:39.537050 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:39.537110 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:39.537437 (+ 327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:39.538195 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:39.538201 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:39.538228 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:39.538379 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.538386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.539876 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.539879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.541519 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.541532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.542820 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.542823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.542936 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.542959 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:29:39.543700 (+ 741us) client_negotiation.cc:240] Negotiation successful
0811 22:29:39.543787 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":207,"thread_start_us":112,"threads_started":1}
I20260811 22:29:39.546625 11775 tablet_service.cc:1467] Processing CreateTablet for tablet 024e04527cfb4bd6afc24ce51eaa48a4 (DEFAULT_TABLE table=test-table [id=252312a86feb48e3ae64574976d2d65a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:39.546623 11466 tablet_service.cc:1467] Processing CreateTablet for tablet 024e04527cfb4bd6afc24ce51eaa48a4 (DEFAULT_TABLE table=test-table [id=252312a86feb48e3ae64574976d2d65a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:39.546732 11618 tablet_service.cc:1467] Processing CreateTablet for tablet 024e04527cfb4bd6afc24ce51eaa48a4 (DEFAULT_TABLE table=test-table [id=252312a86feb48e3ae64574976d2d65a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:39.547761 11775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 024e04527cfb4bd6afc24ce51eaa48a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:39.547761 11466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 024e04527cfb4bd6afc24ce51eaa48a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:39.547768 11618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 024e04527cfb4bd6afc24ce51eaa48a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:39.554298 11873 tablet_bootstrap.cc:492] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: Bootstrap starting.
I20260811 22:29:39.555621 11875 tablet_bootstrap.cc:492] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c: Bootstrap starting.
I20260811 22:29:39.556177 11876 tablet_bootstrap.cc:492] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: Bootstrap starting.
I20260811 22:29:39.556788 11873 tablet_bootstrap.cc:654] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:39.557853 11875 tablet_bootstrap.cc:654] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:39.558183 11876 tablet_bootstrap.cc:654] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:39.558127 11873 log.cc:824] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:39.558770 11875 log.cc:824] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:39.559370 11876 log.cc:824] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:39.560722 11873 tablet_bootstrap.cc:492] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: No bootstrap required, opened a new log
I20260811 22:29:39.560935 11873 ts_tablet_manager.cc:1397] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:29:39.561165 11876 tablet_bootstrap.cc:492] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: No bootstrap required, opened a new log
I20260811 22:29:39.561362 11876 ts_tablet_manager.cc:1397] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:29:39.564276 11873 raft_consensus.cc:348] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } }
I20260811 22:29:39.564287 11876 raft_consensus.cc:348] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } }
I20260811 22:29:39.564564 11873 raft_consensus.cc:374] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:39.564565 11876 raft_consensus.cc:374] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:39.564625 11876 raft_consensus.cc:729] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7427144fa17406e8e7bce8ed543eee8, State: Initialized, Role: FOLLOWER
I20260811 22:29:39.564705 11873 raft_consensus.cc:729] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88b478e6b8404466a9a60acf7ce55001, State: Initialized, Role: FOLLOWER
I20260811 22:29:39.565131 11876 consensus_queue.cc:260] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } }
I20260811 22:29:39.565196 11873 consensus_queue.cc:260] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } }
I20260811 22:29:39.565451 11875 tablet_bootstrap.cc:492] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c: No bootstrap required, opened a new log
I20260811 22:29:39.565670 11875 ts_tablet_manager.cc:1397] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 22:29:39.566314 11876 ts_tablet_manager.cc:1428] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 22:29:39.566342 11873 ts_tablet_manager.cc:1428] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:29:39.566747 11854 heartbeater.cc:503] Master 127.24.71.254:40533 was elected leader, sending a full tablet report...
I20260811 22:29:39.566756 11695 heartbeater.cc:503] Master 127.24.71.254:40533 was elected leader, sending a full tablet report...
I20260811 22:29:39.570398 11875 raft_consensus.cc:348] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } }
I20260811 22:29:39.570739 11875 raft_consensus.cc:374] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:39.570845 11875 raft_consensus.cc:729] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02314ef0b92f4b56a46a55370ddd582c, State: Initialized, Role: FOLLOWER
I20260811 22:29:39.571384 11875 consensus_queue.cc:260] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } }
I20260811 22:29:39.572131 11540 heartbeater.cc:503] Master 127.24.71.254:40533 was elected leader, sending a full tablet report...
I20260811 22:29:39.572381 11875 ts_tablet_manager.cc:1428] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c: Time spent starting tablet: real 0.007s user 0.001s sys 0.006s
I20260811 22:29:39.668076 11882 raft_consensus.cc:482] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:39.668294 11882 raft_consensus.cc:504] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } }
I20260811 22:29:39.669505 11882 leader_election.cc:288] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7427144fa17406e8e7bce8ed543eee8 (127.24.71.195:44215), 88b478e6b8404466a9a60acf7ce55001 (127.24.71.194:38717)
I20260811 22:29:39.674077 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.670054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:51417 (local address 127.24.71.195:44215)
0811 22:29:39.670188 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.670192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.670596 (+ 404us) server_negotiation.cc:407] Connection header received
0811 22:29:39.670798 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:39.670801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:39.670857 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:39.670954 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:39.671247 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.671728 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.672955 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.673471 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.673607 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.673666 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:39.673862 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 22:29:39.673933 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:29:39.674093 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.669923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:38095 (local address 127.24.71.194:38717)
0811 22:29:39.670089 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.670095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.670437 (+ 342us) server_negotiation.cc:407] Connection header received
0811 22:29:39.670593 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:39.670596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:39.670659 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:39.670750 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:39.671189 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.671713 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.673039 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.673478 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.673635 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.673691 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:39.673901 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 22:29:39.673971 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:29:39.674105 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.669823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:38717 (local address 127.24.71.193:38095)
0811 22:29:39.670282 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:39.670295 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:39.670462 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:39.670798 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:39.670801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:39.670822 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:39.671045 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.671057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.671866 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.671871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.672840 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.672850 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.673635 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.673639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.673749 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.673769 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:29:39.673862 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:29:39.673933 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":77,"thread_start_us":144,"threads_started":1}
I20260811 22:29:39.674109 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.669823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:44215 (local address 127.24.71.193:51417)
0811 22:29:39.670436 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:39.670453 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:39.670612 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:39.670981 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:39.670983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:39.670998 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:39.671099 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.671103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.671914 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.671917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.672826 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:39.672835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.673608 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.673612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:39.673721 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.673757 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:29:39.673819 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 22:29:39.673919 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":168,"thread_start_us":81,"threads_started":1}
I20260811 22:29:39.674835 11799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e04527cfb4bd6afc24ce51eaa48a4" candidate_uuid: "02314ef0b92f4b56a46a55370ddd582c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7427144fa17406e8e7bce8ed543eee8" is_pre_election: true
I20260811 22:29:39.674831 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e04527cfb4bd6afc24ce51eaa48a4" candidate_uuid: "02314ef0b92f4b56a46a55370ddd582c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88b478e6b8404466a9a60acf7ce55001" is_pre_election: true
I20260811 22:29:39.675143 11640 raft_consensus.cc:2454] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02314ef0b92f4b56a46a55370ddd582c in term 0.
I20260811 22:29:39.675144 11799 raft_consensus.cc:2454] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02314ef0b92f4b56a46a55370ddd582c in term 0.
I20260811 22:29:39.675650 11417 leader_election.cc:302] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 02314ef0b92f4b56a46a55370ddd582c, 88b478e6b8404466a9a60acf7ce55001, b7427144fa17406e8e7bce8ed543eee8; no voters:
I20260811 22:29:39.676028 11882 raft_consensus.cc:2781] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:29:39.676117 11882 raft_consensus.cc:482] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:29:39.676174 11882 raft_consensus.cc:3037] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:39.677321 11882 raft_consensus.cc:504] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } }
I20260811 22:29:39.677827 11882 leader_election.cc:288] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [CANDIDATE]: Term 1 election: Requested vote from peers b7427144fa17406e8e7bce8ed543eee8 (127.24.71.195:44215), 88b478e6b8404466a9a60acf7ce55001 (127.24.71.194:38717)
I20260811 22:29:39.678268 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e04527cfb4bd6afc24ce51eaa48a4" candidate_uuid: "02314ef0b92f4b56a46a55370ddd582c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88b478e6b8404466a9a60acf7ce55001"
I20260811 22:29:39.678268 11799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "024e04527cfb4bd6afc24ce51eaa48a4" candidate_uuid: "02314ef0b92f4b56a46a55370ddd582c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7427144fa17406e8e7bce8ed543eee8"
I20260811 22:29:39.678424 11640 raft_consensus.cc:3037] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:39.678426 11799 raft_consensus.cc:3037] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:39.679869 11640 raft_consensus.cc:2454] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02314ef0b92f4b56a46a55370ddd582c in term 1.
I20260811 22:29:39.679868 11799 raft_consensus.cc:2454] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02314ef0b92f4b56a46a55370ddd582c in term 1.
I20260811 22:29:39.680256 11417 leader_election.cc:302] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02314ef0b92f4b56a46a55370ddd582c, 88b478e6b8404466a9a60acf7ce55001; no voters:
I20260811 22:29:39.680455 11882 raft_consensus.cc:2781] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:39.680694 11882 raft_consensus.cc:686] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 1 LEADER]: Becoming Leader. State: Replica: 02314ef0b92f4b56a46a55370ddd582c, State: Running, Role: LEADER
I20260811 22:29:39.681059 11882 consensus_queue.cc:237] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } }
I20260811 22:29:39.685045 11324 catalog_manager.cc:5495] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c reported cstate change: term changed from 0 to 1, leader changed from <none> to 02314ef0b92f4b56a46a55370ddd582c (127.24.71.193). New cstate: current_term: 1 leader_uuid: "02314ef0b92f4b56a46a55370ddd582c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02314ef0b92f4b56a46a55370ddd582c" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 } health_report { overall_health: UNKNOWN } } }
I20260811 22:29:39.697237 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.693426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35220 (local address 127.24.71.193:36539)
0811 22:29:39.693600 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.693603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.693621 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:29:39.693713 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:39.693715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:39.693772 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:39.693871 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:29:39.694177 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.694683 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.695589 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:39.695805 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:39.695972 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:39.696279 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:29:39.696804 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:29:39.696913 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:39.697044 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 22:29:39.697114 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 22:29:39.703902 11640 raft_consensus.cc:1261] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [term 1 FOLLOWER]: Refusing update from remote peer 02314ef0b92f4b56a46a55370ddd582c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:39.703902 11799 raft_consensus.cc:1261] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [term 1 FOLLOWER]: Refusing update from remote peer 02314ef0b92f4b56a46a55370ddd582c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:39.704866 11882 consensus_queue.cc:1035] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88b478e6b8404466a9a60acf7ce55001" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:39.705143 11885 consensus_queue.cc:1035] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7427144fa17406e8e7bce8ed543eee8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:39.717075 11888 mvcc.cc:205] Tried to move back new op lower bound from 7317452307258408960 to 7317452307176828928. Current Snapshot: MvccSnapshot[applied={T|T < 7317452307258408960}]
I20260811 22:29:39.726423 11324 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:39.726604 11324 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:39.729532 11324 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbdbb48494da434885584d7574621871: Sending DeleteTablet for 3 replicas of tablet 024e04527cfb4bd6afc24ce51eaa48a4
I20260811 22:29:39.730494 11618 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e04527cfb4bd6afc24ce51eaa48a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:39 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:40620
I20260811 22:29:39.730476 11775 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e04527cfb4bd6afc24ce51eaa48a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:39 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:59496
I20260811 22:29:39.730473 11466 tablet_service.cc:1514] Processing DeleteTablet for tablet 024e04527cfb4bd6afc24ce51eaa48a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:39 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:35216
I20260811 22:29:39.731106 11902 tablet_replica.cc:330] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: stopping tablet replica
I20260811 22:29:39.731240 11904 tablet_replica.cc:330] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c: stopping tablet replica
I20260811 22:29:39.731491 11902 raft_consensus.cc:2229] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:39.731559 11904 raft_consensus.cc:2229] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:29:39.731741 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 11397
I20260811 22:29:39.732012 11904 raft_consensus.cc:2258] T 024e04527cfb4bd6afc24ce51eaa48a4 P 02314ef0b92f4b56a46a55370ddd582c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:39.732357 11901 tablet_replica.cc:330] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: stopping tablet replica
I20260811 22:29:39.732388 11902 raft_consensus.cc:2258] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:39.732646 11901 raft_consensus.cc:2229] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:39.732971 11901 raft_consensus.cc:2258] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:39.734910 11902 ts_tablet_manager.cc:1905] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:39.735060 11901 ts_tablet_manager.cc:1905] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:39.738093 11902 ts_tablet_manager.cc:1918] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:39.738229 11902 log.cc:1196] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/024e04527cfb4bd6afc24ce51eaa48a4
I20260811 22:29:39.738602 11902 ts_tablet_manager.cc:1939] T 024e04527cfb4bd6afc24ce51eaa48a4 P b7427144fa17406e8e7bce8ed543eee8: Deleting consensus metadata
I20260811 22:29:39.738853 11901 ts_tablet_manager.cc:1918] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:39.738970 11901 log.cc:1196] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/024e04527cfb4bd6afc24ce51eaa48a4
I20260811 22:29:39.739334 11901 ts_tablet_manager.cc:1939] T 024e04527cfb4bd6afc24ce51eaa48a4 P 88b478e6b8404466a9a60acf7ce55001: Deleting consensus metadata
I20260811 22:29:39.740083 11306 catalog_manager.cc:4841] TS b7427144fa17406e8e7bce8ed543eee8 (127.24.71.195:44215): tablet 024e04527cfb4bd6afc24ce51eaa48a4 (table test-table [id=252312a86feb48e3ae64574976d2d65a]) successfully deleted
I20260811 22:29:39.740698 11309 catalog_manager.cc:4841] TS 88b478e6b8404466a9a60acf7ce55001 (127.24.71.194:38717): tablet 024e04527cfb4bd6afc24ce51eaa48a4 (table test-table [id=252312a86feb48e3ae64574976d2d65a]) successfully deleted
W20260811 22:29:39.743748 11309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:36539 (error 108)
I20260811 22:29:39.743933 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 11546
I20260811 22:29:39.747416 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.746975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:36539 (local address 127.0.0.1:35230)
0811 22:29:39.747177 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:39.747285 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:36539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 22:29:39.747666 11309 catalog_manager.cc:4584] TS 02314ef0b92f4b56a46a55370ddd582c (127.24.71.193:36539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 024e04527cfb4bd6afc24ce51eaa48a4: Network error: Client connection negotiation failed: client connection to 127.24.71.193:36539: connect: Connection refused (error 111)
I20260811 22:29:39.753173 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 11703
I20260811 22:29:39.761428 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 11287
2026-08-11T22:29:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2641 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T22:29:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:39Z Disabled control of system clock
I20260811 22:29:39.799134 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42139
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:42139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:29:39.919760 11909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:39.920113 11909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:39.920152 11909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:39.924285 11909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:39.924356 11909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:39.924371 11909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:39.924394 11909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:39.924410 11909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:39.929476 11909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:42139
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:39.930693 11909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:39.932173 11909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:39.941723 11916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:39.941722 11915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:39.941722 11918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:39.942432 11909 server_base.cc:1034] running on GCE node
I20260811 22:29:39.943130 11909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:39.944518 11909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:39.945735 11909 hybrid_clock.cc:648] HybridClock initialized: now 1786487379945718 us; error 35 us; skew 500 ppm
I20260811 22:29:39.948016 11909 webserver.cc:460] Webserver started at http://127.24.71.254:35821/ using document root <none> and password file <none>
I20260811 22:29:39.948740 11909 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:39.948848 11909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:39.949127 11909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:39.951448 11909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "9432cd6d3202425d9e12f7e874d0060d"
format_stamp: "Formatted at 2026-08-11 22:29:39 on dist-test-slave-mvvj"
server_key: "c6accb4b089ab0a6ba8e9c41d42d5a02"
server_key_iv: "8600310e755b9817f8fccda762c16506"
server_key_version: "encryptionkey@0"
I20260811 22:29:39.952162 11909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "9432cd6d3202425d9e12f7e874d0060d"
format_stamp: "Formatted at 2026-08-11 22:29:39 on dist-test-slave-mvvj"
server_key: "c6accb4b089ab0a6ba8e9c41d42d5a02"
server_key_iv: "8600310e755b9817f8fccda762c16506"
server_key_version: "encryptionkey@0"
I20260811 22:29:39.956489 11909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 22:29:39.959283 11925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:39.960765 11909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:39.960935 11909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "9432cd6d3202425d9e12f7e874d0060d"
format_stamp: "Formatted at 2026-08-11 22:29:39 on dist-test-slave-mvvj"
server_key: "c6accb4b089ab0a6ba8e9c41d42d5a02"
server_key_iv: "8600310e755b9817f8fccda762c16506"
server_key_version: "encryptionkey@0"
I20260811 22:29:39.961071 11909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:39.977416 11909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:39.978259 11909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:39.978518 11909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:39.986564 11909 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:42139
I20260811 22:29:39.986595 11984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:42139 every 8 connection(s)
I20260811 22:29:39.987703 11909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:39.991369 11985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:39.996536 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 11909
I20260811 22:29:39.996690 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:39.997042 24863 external_mini_cluster.cc:1428] Setting key ec86e16122b09a8c90a4b66bfe077028
I20260811 22:29:39.998425 11985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d: Bootstrap starting.
I20260811 22:29:40.001128 11985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:40.001971 11985 log.cc:824] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:40.004511 11985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d: No bootstrap required, opened a new log
I20260811 22:29:40.008769 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:39.998750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.24.71.254:42139)
0811 22:29:39.999605 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:39.999625 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:39.999668 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:29:40.000558 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.000580 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.001070 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.001541 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:40.002091 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.003542 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.004714 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.005276 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.005512 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.006610 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:40.006616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:40.006633 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:40.007096 (+ 463us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:40.007167 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:40.007191 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:40.007329 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.007497 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.007815 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":171,"threads_started":1}
I20260811 22:29:40.009305 11985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9432cd6d3202425d9e12f7e874d0060d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42139 } }
I20260811 22:29:40.009629 11985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:40.009711 11985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9432cd6d3202425d9e12f7e874d0060d, State: Initialized, Role: FOLLOWER
I20260811 22:29:40.010234 11985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9432cd6d3202425d9e12f7e874d0060d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42139 } }
I20260811 22:29:40.010380 11985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:40.010473 11985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:40.010599 11985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:40.011962 11985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9432cd6d3202425d9e12f7e874d0060d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42139 } }
I20260811 22:29:40.012367 11985 leader_election.cc:302] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9432cd6d3202425d9e12f7e874d0060d; no voters:
I20260811 22:29:40.012776 11985 leader_election.cc:288] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:40.012993 11993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:40.013355 11993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [term 1 LEADER]: Becoming Leader. State: Replica: 9432cd6d3202425d9e12f7e874d0060d, State: Running, Role: LEADER
I20260811 22:29:40.013792 11993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9432cd6d3202425d9e12f7e874d0060d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42139 } }
I20260811 22:29:40.014540 11985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:40.015823 11995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9432cd6d3202425d9e12f7e874d0060d. Latest consensus state: current_term: 1 leader_uuid: "9432cd6d3202425d9e12f7e874d0060d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9432cd6d3202425d9e12f7e874d0060d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42139 } } }
I20260811 22:29:40.015895 11994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9432cd6d3202425d9e12f7e874d0060d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9432cd6d3202425d9e12f7e874d0060d" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42139 } } }
I20260811 22:29:40.016064 11995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:40.016073 11994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:40.016508 12002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:40.020133 12002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:40.027104 12002 catalog_manager.cc:1347] Generated new cluster ID: e613bcf92ebe4b7ab615121e0d38820e
I20260811 22:29:40.027220 12002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:40.039458 12002 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:40.040542 12002 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:40.046412 12002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d: Generated new TSK 0
I20260811 22:29:40.047469 12002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:29:40.052100 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42139
--builtin_ntp_servers=127.24.71.212:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:29:40.177726 12012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:40.178042 12012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:40.178086 12012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:40.182614 12012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:40.182708 12012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:40.182768 12012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:29:40.188163 12012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:40.189450 12012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:40.190932 12012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:40.200824 12022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:40.200908 12012 server_base.cc:1034] running on GCE node
W20260811 22:29:40.200855 12018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:40.200855 12019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:40.201539 12012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:40.202271 12012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:40.203473 12012 hybrid_clock.cc:648] HybridClock initialized: now 1786487380203454 us; error 31 us; skew 500 ppm
I20260811 22:29:40.205678 12012 webserver.cc:460] Webserver started at http://127.24.71.193:35025/ using document root <none> and password file <none>
I20260811 22:29:40.206300 12012 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:40.206398 12012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:40.206650 12012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:40.208904 12012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "466cd48b6dd041a0883965ec0562cfa6"
format_stamp: "Formatted at 2026-08-11 22:29:40 on dist-test-slave-mvvj"
server_key: "346d9352363418a756a0d89f7afd1a41"
server_key_iv: "19ffbb98be3fcb8fce55921887617cf4"
server_key_version: "encryptionkey@0"
I20260811 22:29:40.209504 12012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "466cd48b6dd041a0883965ec0562cfa6"
format_stamp: "Formatted at 2026-08-11 22:29:40 on dist-test-slave-mvvj"
server_key: "346d9352363418a756a0d89f7afd1a41"
server_key_iv: "19ffbb98be3fcb8fce55921887617cf4"
server_key_version: "encryptionkey@0"
I20260811 22:29:40.213883 12012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:29:40.216960 12030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:40.218423 12012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:29:40.218580 12012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "466cd48b6dd041a0883965ec0562cfa6"
format_stamp: "Formatted at 2026-08-11 22:29:40 on dist-test-slave-mvvj"
server_key: "346d9352363418a756a0d89f7afd1a41"
server_key_iv: "19ffbb98be3fcb8fce55921887617cf4"
server_key_version: "encryptionkey@0"
I20260811 22:29:40.218696 12012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:40.230636 12012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:40.231429 12012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:40.231606 12012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:40.232280 12012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:40.233286 12012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:40.233338 12012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:40.233381 12012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:40.233397 12012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:40.243304 12012 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:40347
I20260811 22:29:40.243327 12156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:40347 every 8 connection(s)
I20260811 22:29:40.244362 12012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:29:40.251534 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 12012
I20260811 22:29:40.251684 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:29:40.252106 24863 external_mini_cluster.cc:1428] Setting key 1e47b9781c1e328d7c8af2b550d7306b
I20260811 22:29:40.254426 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42139
--builtin_ntp_servers=127.24.71.212:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:29:40.255106 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.246375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:36817 (local address 127.24.71.254:42139)
0811 22:29:40.246570 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.246576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.247944 (+ 1368us) server_negotiation.cc:407] Connection header received
0811 22:29:40.249458 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.249461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.249521 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.249602 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:40.250554 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.251110 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.252880 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.253136 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.253262 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.254570 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:40.254576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:40.254580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:40.254600 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:40.254632 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:40.254644 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:40.254712 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.254927 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.254984 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 22:29:40.256453 12159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.246911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42139 (local address 127.24.71.193:36817)
0811 22:29:40.247711 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:40.247758 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:40.249214 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:40.249791 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:40.249798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:40.249894 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:40.250397 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.250417 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.251262 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.251268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.252738 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.252749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.253312 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.253316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.253508 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.254194 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:29:40.254198 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:40.254258 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:29:40.254354 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:40.254741 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:40.254754 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:40.254764 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:29:40.254909 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 22:29:40.255255 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":124,"threads_started":1}
I20260811 22:29:40.257863 12157 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42139
I20260811 22:29:40.258141 12157 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:40.258826 12157 heartbeater.cc:511] Master 127.24.71.254:42139 requested a full tablet report, sending...
I20260811 22:29:40.260763 11943 ts_manager.cc:194] Registered new tserver with Master: 466cd48b6dd041a0883965ec0562cfa6 (127.24.71.193:40347)
I20260811 22:29:40.262583 11943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.193:36817
W20260811 22:29:40.378584 12161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:40.379032 12161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:40.379106 12161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:40.383535 12161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:40.383605 12161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:40.383666 12161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:29:40.388953 12161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:40.390203 12161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:40.391698 12161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:40.402530 12168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:40.402495 12171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:40.402675 12161 server_base.cc:1034] running on GCE node
W20260811 22:29:40.402495 12169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:40.403339 12161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:40.404181 12161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:40.405364 12161 hybrid_clock.cc:648] HybridClock initialized: now 1786487380405347 us; error 32 us; skew 500 ppm
I20260811 22:29:40.407595 12161 webserver.cc:460] Webserver started at http://127.24.71.194:35527/ using document root <none> and password file <none>
I20260811 22:29:40.408332 12161 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:40.408408 12161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:40.408706 12161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:40.411059 12161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "2f949814da0a4550999f2c056e65cee5"
format_stamp: "Formatted at 2026-08-11 22:29:40 on dist-test-slave-mvvj"
server_key: "6b8545881993e1e4c0ef0d332eb781ba"
server_key_iv: "a1249baceda1a2c74db1058b8ea649d9"
server_key_version: "encryptionkey@0"
I20260811 22:29:40.411697 12161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "2f949814da0a4550999f2c056e65cee5"
format_stamp: "Formatted at 2026-08-11 22:29:40 on dist-test-slave-mvvj"
server_key: "6b8545881993e1e4c0ef0d332eb781ba"
server_key_iv: "a1249baceda1a2c74db1058b8ea649d9"
server_key_version: "encryptionkey@0"
I20260811 22:29:40.416111 12161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:29:40.418983 12178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:40.420449 12161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:40.420614 12161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "2f949814da0a4550999f2c056e65cee5"
format_stamp: "Formatted at 2026-08-11 22:29:40 on dist-test-slave-mvvj"
server_key: "6b8545881993e1e4c0ef0d332eb781ba"
server_key_iv: "a1249baceda1a2c74db1058b8ea649d9"
server_key_version: "encryptionkey@0"
I20260811 22:29:40.420755 12161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:40.438732 12161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:40.439666 12161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:40.439930 12161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:40.440799 12161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:40.442070 12161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:40.442128 12161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:40.442174 12161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:40.442193 12161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:40.452526 12161 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:40051
I20260811 22:29:40.452618 12306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:40051 every 8 connection(s)
I20260811 22:29:40.453615 12161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:29:40.463665 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 12161
I20260811 22:29:40.463824 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:29:40.464035 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.455670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:34141 (local address 127.24.71.254:42139)
0811 22:29:40.455863 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.455868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.457121 (+ 1253us) server_negotiation.cc:407] Connection header received
0811 22:29:40.458589 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.458591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.458680 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.458800 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:40.459923 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.460509 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.461708 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.461949 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.462046 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.463359 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:40.463362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:40.463365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:40.463380 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:40.463402 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:40.463411 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:40.463481 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.463734 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.463839 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:29:40.464200 24863 external_mini_cluster.cc:1428] Setting key 41af6fa233b9cbceeac52719049dab90
I20260811 22:29:40.465135 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.456203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42139 (local address 127.24.71.194:34141)
0811 22:29:40.456945 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:40.456990 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:40.458360 (+ 1370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:40.459015 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:40.459023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:40.459125 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:40.459699 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.459715 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.460654 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.460657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.461574 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.461584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.462121 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.462124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.462274 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.463013 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:29:40.463015 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:40.463060 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:29:40.463135 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:40.463540 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:40.463552 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:40.463560 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:29:40.463726 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 22:29:40.464101 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":131,"threads_started":1}
I20260811 22:29:40.466567 12307 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42139
I20260811 22:29:40.466609 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42139
--builtin_ntp_servers=127.24.71.212:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:29:40.466845 12307 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:40.467526 12307 heartbeater.cc:511] Master 127.24.71.254:42139 requested a full tablet report, sending...
I20260811 22:29:40.468889 11943 ts_manager.cc:194] Registered new tserver with Master: 2f949814da0a4550999f2c056e65cee5 (127.24.71.194:40051)
I20260811 22:29:40.469558 11943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.194:34141
W20260811 22:29:40.600458 12310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:40.600730 12310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:40.600771 12310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:40.604876 12310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:40.604938 12310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:40.604997 12310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:29:40.610069 12310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:42139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:40.611279 12310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:40.612862 12310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:40.624154 12319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:40.624213 12317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:40.624154 12316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:40.624591 12310 server_base.cc:1034] running on GCE node
I20260811 22:29:40.625216 12310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:40.626099 12310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:40.627290 12310 hybrid_clock.cc:648] HybridClock initialized: now 1786487380627264 us; error 38 us; skew 500 ppm
I20260811 22:29:40.629766 12310 webserver.cc:460] Webserver started at http://127.24.71.195:46573/ using document root <none> and password file <none>
I20260811 22:29:40.630463 12310 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:40.630540 12310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:40.630786 12310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:40.633183 12310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d"
format_stamp: "Formatted at 2026-08-11 22:29:40 on dist-test-slave-mvvj"
server_key: "2ffc456900758404673e15a2b42f19e9"
server_key_iv: "aec636edc1ddec88104a23df4f4ffda7"
server_key_version: "encryptionkey@0"
I20260811 22:29:40.633834 12310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d"
format_stamp: "Formatted at 2026-08-11 22:29:40 on dist-test-slave-mvvj"
server_key: "2ffc456900758404673e15a2b42f19e9"
server_key_iv: "aec636edc1ddec88104a23df4f4ffda7"
server_key_version: "encryptionkey@0"
I20260811 22:29:40.637941 12310 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:29:40.640825 12326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:40.642328 12310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:40.642514 12310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d"
format_stamp: "Formatted at 2026-08-11 22:29:40 on dist-test-slave-mvvj"
server_key: "2ffc456900758404673e15a2b42f19e9"
server_key_iv: "aec636edc1ddec88104a23df4f4ffda7"
server_key_version: "encryptionkey@0"
I20260811 22:29:40.642665 12310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:40.672170 12310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:40.673127 12310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:40.673386 12310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:40.674165 12310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:40.675397 12310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:40.675503 12310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:40.675613 12310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:40.675657 12310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:40.686816 12310 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:45111
I20260811 22:29:40.686816 12454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:45111 every 8 connection(s)
I20260811 22:29:40.688624 12310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:29:40.696163 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 12310
I20260811 22:29:40.696313 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:29:40.696683 24863 external_mini_cluster.cc:1428] Setting key 05d66f432a5fae2e4d143f889e0533c3
I20260811 22:29:40.700727 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.691020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:35633 (local address 127.24.71.254:42139)
0811 22:29:40.691198 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.691202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.692461 (+ 1259us) server_negotiation.cc:407] Connection header received
0811 22:29:40.693927 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.693930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.693996 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.694110 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:40.695375 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.696144 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.698335 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.698612 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.698751 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.700099 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:40.700103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:40.700106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:40.700127 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:40.700150 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:40.700158 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:40.700243 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.700500 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.700577 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:29:40.701747 12457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.691340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:42139 (local address 127.24.71.195:35633)
0811 22:29:40.692206 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:40.692256 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:40.693629 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:40.694282 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:40.694290 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:40.694489 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:40.695164 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.695182 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.696462 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.696466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.698151 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.698166 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.698805 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.698810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.699032 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.699699 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:29:40.699700 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:40.699748 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:29:40.699860 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:40.700260 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:40.700271 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:40.700278 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:29:40.700492 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 22:29:40.700857 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":150,"threads_started":1}
I20260811 22:29:40.703122 12455 heartbeater.cc:347] Connected to a master server at 127.24.71.254:42139
I20260811 22:29:40.703413 12455 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:40.704082 12455 heartbeater.cc:511] Master 127.24.71.254:42139 requested a full tablet report, sending...
I20260811 22:29:40.705268 11943 ts_manager.cc:194] Registered new tserver with Master: c18e59b1cd4c4e0aae62e2dd34b9474d (127.24.71.195:45111)
I20260811 22:29:40.705875 11943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.195:35633
I20260811 22:29:40.711393 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:29:40.719074 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.713809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.24.71.254:42139)
0811 22:29:40.714002 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.714006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.714197 (+ 191us) server_negotiation.cc:407] Connection header received
0811 22:29:40.714276 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.714278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.714334 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.714475 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:40.714858 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.715613 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.716824 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.717194 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.717296 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.718506 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:40.718510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:40.718512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:40.718538 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:40.718561 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:40.718570 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:40.718656 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.718855 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.718929 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 22:29:40.723114 11943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:29:40.726595 11943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:29:40.743152 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.736517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:45111 (local address 127.0.0.1:40878)
0811 22:29:40.737539 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:40.737553 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:40.737635 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:40.738351 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:40.738355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:40.738378 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:40.738498 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.738505 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.739772 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.739775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.741469 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.741481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.742661 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.742665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.742766 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.742790 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:29:40.742861 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:29:40.742948 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"mutex_wait_us":78,"thread_start_us":111,"threads_started":1}
I20260811 22:29:40.743156 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.736635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49248 (local address 127.24.71.194:40051)
0811 22:29:40.737589 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.737603 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.737629 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:29:40.737715 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.737723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.737948 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.738529 (+ 581us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:40.738694 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.739911 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.741626 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.742291 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.742435 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.742675 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.742835 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.742949 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":153,"threads_started":1}
I20260811 22:29:40.743155 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.736465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:40051 (local address 127.0.0.1:49248)
0811 22:29:40.737060 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:40.737123 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:40.737282 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:40.738142 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:40.738148 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:40.738170 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:40.738366 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.738375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.740028 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.740031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.741497 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.741504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.742440 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.742444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.742591 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.742637 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:29:40.742797 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 22:29:40.742949 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":104,"thread_start_us":134,"threads_started":1}
I20260811 22:29:40.743738 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.737776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.24.71.193:40347)
0811 22:29:40.738162 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.738170 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.738434 (+ 264us) server_negotiation.cc:407] Connection header received
0811 22:29:40.738659 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.738666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.738846 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.739005 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:40.739230 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.740301 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.741676 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.742512 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.742635 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.742824 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.743228 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.743571 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":98,"threads_started":1}
I20260811 22:29:40.743991 12471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.737470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40878 (local address 127.24.71.195:45111)
0811 22:29:40.737893 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.737903 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.737927 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:29:40.738004 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.738012 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.738198 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.738367 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:40.738678 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.739644 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.741639 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.742512 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.743412 (+ 900us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.743613 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.743668 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.743764 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":154,"threads_started":1}
I20260811 22:29:40.744132 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.737324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40347 (local address 127.0.0.1:59164)
0811 22:29:40.738336 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:40.738349 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:40.738499 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:40.738998 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:40.739002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:40.739022 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:40.739106 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.739112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.740433 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.740437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.741510 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.741540 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.742885 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.742888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.742965 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.742986 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:29:40.743054 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 22:29:40.743111 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":21,"thread_start_us":103,"threads_started":1}
I20260811 22:29:40.746793 12233 tablet_service.cc:1467] Processing CreateTablet for tablet ce39753271ee46b48ea4bdd48c304b72 (DEFAULT_TABLE table=test-table [id=c5cfb48edadf4bf5860d77243352032b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:40.747108 12382 tablet_service.cc:1467] Processing CreateTablet for tablet ce39753271ee46b48ea4bdd48c304b72 (DEFAULT_TABLE table=test-table [id=c5cfb48edadf4bf5860d77243352032b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:40.748095 12233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce39753271ee46b48ea4bdd48c304b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:40.748354 12382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce39753271ee46b48ea4bdd48c304b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:40.750242 12084 tablet_service.cc:1467] Processing CreateTablet for tablet ce39753271ee46b48ea4bdd48c304b72 (DEFAULT_TABLE table=test-table [id=c5cfb48edadf4bf5860d77243352032b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:40.751410 12084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce39753271ee46b48ea4bdd48c304b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:40.756562 12474 tablet_bootstrap.cc:492] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: Bootstrap starting.
I20260811 22:29:40.758746 12474 tablet_bootstrap.cc:654] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:40.758805 12476 tablet_bootstrap.cc:492] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6: Bootstrap starting.
I20260811 22:29:40.759483 12473 tablet_bootstrap.cc:492] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: Bootstrap starting.
I20260811 22:29:40.759757 12474 log.cc:824] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:40.761184 12476 tablet_bootstrap.cc:654] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:40.761734 12473 tablet_bootstrap.cc:654] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:40.762105 12476 log.cc:824] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:40.762382 12474 tablet_bootstrap.cc:492] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: No bootstrap required, opened a new log
I20260811 22:29:40.762604 12473 log.cc:824] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:40.762665 12474 ts_tablet_manager.cc:1397] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:29:40.764302 12476 tablet_bootstrap.cc:492] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6: No bootstrap required, opened a new log
I20260811 22:29:40.764541 12476 ts_tablet_manager.cc:1397] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:29:40.765687 12473 tablet_bootstrap.cc:492] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: No bootstrap required, opened a new log
I20260811 22:29:40.765900 12473 ts_tablet_manager.cc:1397] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:29:40.768054 12476 raft_consensus.cc:348] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } }
I20260811 22:29:40.768324 12476 raft_consensus.cc:374] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:40.768392 12476 raft_consensus.cc:729] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466cd48b6dd041a0883965ec0562cfa6, State: Initialized, Role: FOLLOWER
I20260811 22:29:40.768478 12474 raft_consensus.cc:348] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } }
I20260811 22:29:40.768743 12474 raft_consensus.cc:374] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:40.768803 12474 raft_consensus.cc:729] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f949814da0a4550999f2c056e65cee5, State: Initialized, Role: FOLLOWER
I20260811 22:29:40.768934 12476 consensus_queue.cc:260] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } }
I20260811 22:29:40.769237 12474 consensus_queue.cc:260] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } }
I20260811 22:29:40.769791 12157 heartbeater.cc:503] Master 127.24.71.254:42139 was elected leader, sending a full tablet report...
I20260811 22:29:40.770156 12476 ts_tablet_manager.cc:1428] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:29:40.770231 12474 ts_tablet_manager.cc:1428] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:29:40.770246 12473 raft_consensus.cc:348] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } }
I20260811 22:29:40.770485 12473 raft_consensus.cc:374] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:40.770535 12473 raft_consensus.cc:729] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c18e59b1cd4c4e0aae62e2dd34b9474d, State: Initialized, Role: FOLLOWER
I20260811 22:29:40.770618 12307 heartbeater.cc:503] Master 127.24.71.254:42139 was elected leader, sending a full tablet report...
I20260811 22:29:40.770962 12473 consensus_queue.cc:260] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } }
I20260811 22:29:40.771973 12473 ts_tablet_manager.cc:1428] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:29:40.772006 12455 heartbeater.cc:503] Master 127.24.71.254:42139 was elected leader, sending a full tablet report...
I20260811 22:29:40.896981 12482 raft_consensus.cc:482] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:40.897260 12482 raft_consensus.cc:504] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } }
I20260811 22:29:40.898648 12482 leader_election.cc:288] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c18e59b1cd4c4e0aae62e2dd34b9474d (127.24.71.195:45111), 466cd48b6dd041a0883965ec0562cfa6 (127.24.71.193:40347)
I20260811 22:29:40.902505 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.899116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:41503 (local address 127.24.71.193:40347)
0811 22:29:40.899256 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.899263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.899282 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:29:40.899371 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.899374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.899474 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.899580 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:40.899893 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.900405 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.901482 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.901922 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.902060 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.902113 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.902301 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.902377 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:29:40.902504 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.898939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40347 (local address 127.24.71.194:41503)
0811 22:29:40.899154 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:40.899169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:40.899272 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:40.899598 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:40.899601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:40.899620 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:40.899746 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.899753 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.900537 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.900542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.901334 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.901344 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.902057 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.902061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.902171 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.902190 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:29:40.902281 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:29:40.902358 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260811 22:29:40.902905 12471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.899046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:40785 (local address 127.24.71.195:45111)
0811 22:29:40.899163 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.899169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.899512 (+ 343us) server_negotiation.cc:407] Connection header received
0811 22:29:40.899664 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.899667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.899722 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.899858 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:40.900227 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.900753 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.901805 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.902223 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.902364 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.902411 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.902703 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.902785 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:29:40.903303 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.898959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:45111 (local address 127.24.71.194:40785)
0811 22:29:40.899413 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:40.899424 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:40.899539 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:40.899851 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:40.899854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:40.899872 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:40.900064 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.900076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.900875 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.900880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.901666 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:40.901675 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.902363 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.902367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:40.902517 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.902536 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:29:40.903072 (+ 536us) client_negotiation.cc:240] Negotiation successful
0811 22:29:40.903152 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":158,"threads_started":1}
I20260811 22:29:40.903650 12107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce39753271ee46b48ea4bdd48c304b72" candidate_uuid: "2f949814da0a4550999f2c056e65cee5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466cd48b6dd041a0883965ec0562cfa6" is_pre_election: true
I20260811 22:29:40.904074 12107 raft_consensus.cc:2454] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f949814da0a4550999f2c056e65cee5 in term 0.
I20260811 22:29:40.904057 12406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce39753271ee46b48ea4bdd48c304b72" candidate_uuid: "2f949814da0a4550999f2c056e65cee5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" is_pre_election: true
I20260811 22:29:40.904338 12406 raft_consensus.cc:2454] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f949814da0a4550999f2c056e65cee5 in term 0.
I20260811 22:29:40.904690 12185 leader_election.cc:302] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f949814da0a4550999f2c056e65cee5, 466cd48b6dd041a0883965ec0562cfa6; no voters:
I20260811 22:29:40.905100 12482 raft_consensus.cc:2781] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:29:40.905184 12482 raft_consensus.cc:482] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:29:40.905215 12482 raft_consensus.cc:3037] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:40.906345 12482 raft_consensus.cc:504] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } }
I20260811 22:29:40.906767 12482 leader_election.cc:288] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [CANDIDATE]: Term 1 election: Requested vote from peers c18e59b1cd4c4e0aae62e2dd34b9474d (127.24.71.195:45111), 466cd48b6dd041a0883965ec0562cfa6 (127.24.71.193:40347)
I20260811 22:29:40.907222 12107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce39753271ee46b48ea4bdd48c304b72" candidate_uuid: "2f949814da0a4550999f2c056e65cee5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "466cd48b6dd041a0883965ec0562cfa6"
I20260811 22:29:40.907222 12406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce39753271ee46b48ea4bdd48c304b72" candidate_uuid: "2f949814da0a4550999f2c056e65cee5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d"
I20260811 22:29:40.907369 12406 raft_consensus.cc:3037] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:40.907369 12107 raft_consensus.cc:3037] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:40.908828 12406 raft_consensus.cc:2454] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f949814da0a4550999f2c056e65cee5 in term 1.
I20260811 22:29:40.908828 12107 raft_consensus.cc:2454] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f949814da0a4550999f2c056e65cee5 in term 1.
I20260811 22:29:40.909185 12185 leader_election.cc:302] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f949814da0a4550999f2c056e65cee5, 466cd48b6dd041a0883965ec0562cfa6; no voters:
I20260811 22:29:40.909454 12482 raft_consensus.cc:2781] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:40.909705 12482 raft_consensus.cc:686] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f949814da0a4550999f2c056e65cee5, State: Running, Role: LEADER
I20260811 22:29:40.910104 12482 consensus_queue.cc:237] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } }
I20260811 22:29:40.913748 11943 catalog_manager.cc:5495] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f949814da0a4550999f2c056e65cee5 (127.24.71.194). New cstate: current_term: 1 leader_uuid: "2f949814da0a4550999f2c056e65cee5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f949814da0a4550999f2c056e65cee5" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 40051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 } health_report { overall_health: UNKNOWN } } }
W20260811 22:29:40.940047 12456 tablet.cc:2369] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:29:40.944202 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.940241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.24.71.194:40051)
0811 22:29:40.940391 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:40.940396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:40.940497 (+ 101us) server_negotiation.cc:407] Connection header received
0811 22:29:40.940722 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:40.940725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:40.940790 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:40.940911 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:29:40.941177 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.941709 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.942622 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:40.942852 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:40.942965 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:40.943301 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:29:40.943745 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:29:40.943917 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:40.943980 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 22:29:40.944045 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:29:40.951102 12107 raft_consensus.cc:1261] T ce39753271ee46b48ea4bdd48c304b72 P 466cd48b6dd041a0883965ec0562cfa6 [term 1 FOLLOWER]: Refusing update from remote peer 2f949814da0a4550999f2c056e65cee5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:40.951102 12406 raft_consensus.cc:1261] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [term 1 FOLLOWER]: Refusing update from remote peer 2f949814da0a4550999f2c056e65cee5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:40.952064 12482 consensus_queue.cc:1035] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c18e59b1cd4c4e0aae62e2dd34b9474d" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:40.952419 12485 consensus_queue.cc:1035] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "466cd48b6dd041a0883965ec0562cfa6" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:29:40.954852 12308 tablet.cc:2369] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:29:40.962911 12489 mvcc.cc:205] Tried to move back new op lower bound from 7317452312367099904 to 7317452312210579456. Current Snapshot: MvccSnapshot[applied={T|T < 7317452312367099904}]
I20260811 22:29:40.970080 11943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:40.970268 11943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:40.973145 11943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9432cd6d3202425d9e12f7e874d0060d: Sending DeleteTablet for 3 replicas of tablet ce39753271ee46b48ea4bdd48c304b72
I20260811 22:29:40.974255 12233 tablet_service.cc:1514] Processing DeleteTablet for tablet ce39753271ee46b48ea4bdd48c304b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:40 UTC) from {username='slave'} at 127.0.0.1:49248
I20260811 22:29:40.974403 12382 tablet_service.cc:1514] Processing DeleteTablet for tablet ce39753271ee46b48ea4bdd48c304b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:40 UTC) from {username='slave'} at 127.0.0.1:40878
I20260811 22:29:40.974874 12497 tablet_replica.cc:330] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: stopping tablet replica
I20260811 22:29:40.974923 12498 tablet_replica.cc:330] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: stopping tablet replica
I20260811 22:29:40.974953 12084 tablet_service.cc:1514] Processing DeleteTablet for tablet ce39753271ee46b48ea4bdd48c304b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:40 UTC) from {username='slave'} at 127.0.0.1:59164
I20260811 22:29:40.975237 12497 raft_consensus.cc:2229] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:29:40.975236 12498 raft_consensus.cc:2229] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:40.975561 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 12012
I20260811 22:29:40.975574 12498 raft_consensus.cc:2258] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:40.975607 12497 raft_consensus.cc:2258] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:40.980917 12497 ts_tablet_manager.cc:1905] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:40.981063 12498 ts_tablet_manager.cc:1905] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:40.984058 12497 ts_tablet_manager.cc:1918] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:40.984058 12498 ts_tablet_manager.cc:1918] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:40.984184 12497 log.cc:1196] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/ce39753271ee46b48ea4bdd48c304b72
I20260811 22:29:40.984184 12498 log.cc:1196] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/ce39753271ee46b48ea4bdd48c304b72
I20260811 22:29:40.984529 12498 ts_tablet_manager.cc:1939] T ce39753271ee46b48ea4bdd48c304b72 P c18e59b1cd4c4e0aae62e2dd34b9474d: Deleting consensus metadata
I20260811 22:29:40.984529 12497 ts_tablet_manager.cc:1939] T ce39753271ee46b48ea4bdd48c304b72 P 2f949814da0a4550999f2c056e65cee5: Deleting consensus metadata
I20260811 22:29:40.986142 11926 catalog_manager.cc:4841] TS c18e59b1cd4c4e0aae62e2dd34b9474d (127.24.71.195:45111): tablet ce39753271ee46b48ea4bdd48c304b72 (table test-table [id=c5cfb48edadf4bf5860d77243352032b]) successfully deleted
I20260811 22:29:40.986459 11928 catalog_manager.cc:4841] TS 2f949814da0a4550999f2c056e65cee5 (127.24.71.194:40051): tablet ce39753271ee46b48ea4bdd48c304b72 (table test-table [id=c5cfb48edadf4bf5860d77243352032b]) successfully deleted
W20260811 22:29:40.992203 11928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:40347 (error 108)
I20260811 22:29:40.993355 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:40.993037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40347 (local address 127.0.0.1:59166)
0811 22:29:40.993183 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:40.993259 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:40347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 22:29:40.993443 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 12161
W20260811 22:29:40.993657 11928 catalog_manager.cc:4584] TS 466cd48b6dd041a0883965ec0562cfa6 (127.24.71.193:40347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce39753271ee46b48ea4bdd48c304b72: Network error: Client connection negotiation failed: client connection to 127.24.71.193:40347: connect: Connection refused (error 111)
I20260811 22:29:41.004027 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 12310
I20260811 22:29:41.013546 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 11909
2026-08-11T22:29:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1251 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:41 dist-test-slave-mvvj krb5kdc[12503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:41 dist-test-slave-mvvj krb5kdc[12503](info): set up 2 sockets
Aug 11 22:29:41 dist-test-slave-mvvj krb5kdc[12503](info): commencing operation
krb5kdc: starting...
W20260811 22:29:42.154224 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:42 dist-test-slave-mvvj krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:29:42.303264 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:34995
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:34995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:42.423151 12518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:42.423517 12518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:42.423607 12518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:42.427927 12518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:42.428042 12518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:42.428078 12518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:42.428119 12518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:42.428153 12518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:42.433936 12518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:34995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:34995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:42.435164 12518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:42.436302 12518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:42.445828 12523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:42.445876 12524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:42.446267 12526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:42.446249 12518 server_base.cc:1034] running on GCE node
I20260811 22:29:42.447361 12518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:42.449144 12518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:42.450348 12518 hybrid_clock.cc:648] HybridClock initialized: now 1786487382450325 us; error 42 us; skew 500 ppm
Aug 11 22:29:42 dist-test-slave-mvvj krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487382, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:42.457921 12518 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:29:42.459100 12518 webserver.cc:460] Webserver started at http://127.24.71.254:40507/ using document root <none> and password file <none>
I20260811 22:29:42.459777 12518 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:42.459966 12518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:42.460249 12518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:42.462775 12518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "2b24c8eb91c844c0a32cf90ad1af8836"
format_stamp: "Formatted at 2026-08-11 22:29:42 on dist-test-slave-mvvj"
server_key: "30123bcd775dea9ea746c89e7dc09a86"
server_key_iv: "6a5d8e6a55359d4055cc2ffe1ecad317"
server_key_version: "encryptionkey@0"
I20260811 22:29:42.463469 12518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "2b24c8eb91c844c0a32cf90ad1af8836"
format_stamp: "Formatted at 2026-08-11 22:29:42 on dist-test-slave-mvvj"
server_key: "30123bcd775dea9ea746c89e7dc09a86"
server_key_iv: "6a5d8e6a55359d4055cc2ffe1ecad317"
server_key_version: "encryptionkey@0"
I20260811 22:29:42.467976 12518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.004s
I20260811 22:29:42.470850 12536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:42.472384 12518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:42.472564 12518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "2b24c8eb91c844c0a32cf90ad1af8836"
format_stamp: "Formatted at 2026-08-11 22:29:42 on dist-test-slave-mvvj"
server_key: "30123bcd775dea9ea746c89e7dc09a86"
server_key_iv: "6a5d8e6a55359d4055cc2ffe1ecad317"
server_key_version: "encryptionkey@0"
I20260811 22:29:42.472708 12518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:42.510481 12518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:42.514092 12518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:42.514369 12518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:42.523386 12518 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:34995
I20260811 22:29:42.523408 12592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:34995 every 8 connection(s)
I20260811 22:29:42.524602 12518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:42.527688 12593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:42.532255 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 12518
I20260811 22:29:42.532416 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:42.532723 24863 external_mini_cluster.cc:1428] Setting key 1a3811e75d77c0b48d6ce2b457eab0ac
I20260811 22:29:42.534313 12593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836: Bootstrap starting.
I20260811 22:29:42.536880 12593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:42.537698 12593 log.cc:824] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:42.539789 12593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836: No bootstrap required, opened a new log
Aug 11 22:29:42 dist-test-slave-mvvj krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:42.544051 12593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b24c8eb91c844c0a32cf90ad1af8836" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34995 } }
I20260811 22:29:42.544418 12593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:42.544508 12593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b24c8eb91c844c0a32cf90ad1af8836, State: Initialized, Role: FOLLOWER
I20260811 22:29:42.545089 12593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b24c8eb91c844c0a32cf90ad1af8836" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34995 } }
I20260811 22:29:42.545258 12593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:42.545356 12593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:42.545475 12593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:42.546465 12593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b24c8eb91c844c0a32cf90ad1af8836" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34995 } }
I20260811 22:29:42.546860 12593 leader_election.cc:302] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b24c8eb91c844c0a32cf90ad1af8836; no voters:
I20260811 22:29:42.547242 12593 leader_election.cc:288] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:42.547328 12598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:42.547578 12598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [term 1 LEADER]: Becoming Leader. State: Replica: 2b24c8eb91c844c0a32cf90ad1af8836, State: Running, Role: LEADER
I20260811 22:29:42.547919 12598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b24c8eb91c844c0a32cf90ad1af8836" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34995 } }
I20260811 22:29:42.548755 12593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:42.549193 12598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b24c8eb91c844c0a32cf90ad1af8836" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b24c8eb91c844c0a32cf90ad1af8836" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34995 } } }
I20260811 22:29:42.549317 12598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:42.549746 12605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:42.549865 12599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b24c8eb91c844c0a32cf90ad1af8836. Latest consensus state: current_term: 1 leader_uuid: "2b24c8eb91c844c0a32cf90ad1af8836" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b24c8eb91c844c0a32cf90ad1af8836" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34995 } } }
I20260811 22:29:42.550061 12599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:42.550484 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:42.534383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.24.71.254:34995)
0811 22:29:42.534921 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:42.534940 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:42.534982 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:29:42.535772 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:42.535794 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:42.536225 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:42.536635 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:42.537611 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:42.538799 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:42.539819 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:42.540465 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:42.540654 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:42.543612 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:42.543619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:42.543637 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:42.543671 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:42.546034 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:42.546565 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:42.546580 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:42.546585 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:42.546767 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:42.547114 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:42.547118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:42.547120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:42.547752 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:42.548695 (+ 943us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:42.548862 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 22:29:42.549226 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":156,"threads_started":1}
I20260811 22:29:42.554324 12605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:42.560354 12605 catalog_manager.cc:1347] Generated new cluster ID: 7703f913a9474fc39f9e6e0b5de764f8
I20260811 22:29:42.560447 12605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:42.584386 12605 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:42.585938 12605 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:42.594409 12605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836: Generated new TSK 0
I20260811 22:29:42.595360 12605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:29:42.639611 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:34995
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:42.762198 12626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:42.762498 12626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:42.762543 12626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:42.766889 12626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:42.766955 12626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:42.767048 12626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:29:42.772426 12626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:34995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:42.773576 12626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:42.774659 12626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:42.785012 12631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:42.785045 12632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:42.785022 12634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:42.785502 12626 server_base.cc:1034] running on GCE node
I20260811 22:29:42.786104 12626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:42.786847 12626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:42.788049 12626 hybrid_clock.cc:648] HybridClock initialized: now 1786487382788028 us; error 41 us; skew 500 ppm
Aug 11 22:29:42 dist-test-slave-mvvj krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487382, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:42.791303 12626 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:29:42.792438 12626 webserver.cc:460] Webserver started at http://127.24.71.193:44979/ using document root <none> and password file <none>
I20260811 22:29:42.793108 12626 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:42.793216 12626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:42.793497 12626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:42.796082 12626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "d6f1d179eff74ee7a40366b26e220f9d"
format_stamp: "Formatted at 2026-08-11 22:29:42 on dist-test-slave-mvvj"
server_key: "9b96d6b5e483fa9ff6c82648cd709d49"
server_key_iv: "e13ca762a319ecaec815acd566d842c0"
server_key_version: "encryptionkey@0"
I20260811 22:29:42.796763 12626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "d6f1d179eff74ee7a40366b26e220f9d"
format_stamp: "Formatted at 2026-08-11 22:29:42 on dist-test-slave-mvvj"
server_key: "9b96d6b5e483fa9ff6c82648cd709d49"
server_key_iv: "e13ca762a319ecaec815acd566d842c0"
server_key_version: "encryptionkey@0"
I20260811 22:29:42.801158 12626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:29:42.803927 12642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:42.805686 12626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:42.805898 12626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "d6f1d179eff74ee7a40366b26e220f9d"
format_stamp: "Formatted at 2026-08-11 22:29:42 on dist-test-slave-mvvj"
server_key: "9b96d6b5e483fa9ff6c82648cd709d49"
server_key_iv: "e13ca762a319ecaec815acd566d842c0"
server_key_version: "encryptionkey@0"
I20260811 22:29:42.806053 12626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:42.823135 12626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:42.826788 12626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:42.827059 12626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:42.827812 12626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:42.828843 12626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:42.828925 12626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:42.829010 12626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:42.829058 12626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:42.840731 12626 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:37015
I20260811 22:29:42.840755 12765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:37015 every 8 connection(s)
I20260811 22:29:42.841796 12626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:29:42.848397 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 12626
I20260811 22:29:42.848706 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:29:42.849087 24863 external_mini_cluster.cc:1428] Setting key b1bcfc9fcea9d0b5dce20c62e75ab763
Aug 11 22:29:42 dist-test-slave-mvvj krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487382, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:42.858830 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:42.844067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:53265 (local address 127.24.71.254:34995)
0811 22:29:42.844255 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:42.844261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:42.845306 (+ 1045us) server_negotiation.cc:407] Connection header received
0811 22:29:42.846779 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:42.846782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:42.846838 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:42.846926 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:42.848473 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:42.849190 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:42.850720 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:42.850978 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:42.851111 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:42.854169 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:42.854181 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:42.854186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:42.854224 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:42.856073 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:42.856927 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:42.856931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:42.856932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:42.857027 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:42.857449 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:42.857499 (+ 50us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:42.857500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:42.857713 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:42.857839 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:42.858483 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 22:29:42.858640 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:29:42.859787 12769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:42.844534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:34995 (local address 127.24.71.193:53265)
0811 22:29:42.845118 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:42.845168 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:42.846511 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:42.847157 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:42.847165 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:42.847607 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:42.848204 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:42.848221 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:42.849383 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:42.849388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:42.850580 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:42.850593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:42.851164 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:42.851170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:42.851366 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:42.852106 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:42.852125 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:42.853906 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:42.856228 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:42.856238 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:42.856262 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:42.856759 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:42.857157 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:42.857161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:42.857163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:42.857330 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:42.857840 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:42.857848 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:42.858183 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 22:29:42.858405 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 22:29:42.858791 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":146,"threads_started":1}
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:29:42.861415 12766 heartbeater.cc:347] Connected to a master server at 127.24.71.254:34995
I20260811 22:29:42.861709 12766 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:42.862309 12766 heartbeater.cc:511] Master 127.24.71.254:34995 requested a full tablet report, sending...
I20260811 22:29:42.864526 12554 ts_manager.cc:194] Registered new tserver with Master: d6f1d179eff74ee7a40366b26e220f9d (127.24.71.193:37015)
I20260811 22:29:42.865957 12554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:53265
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:29:42.888477 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:34995
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:43.014781 12772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:43.015138 12772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:43.015187 12772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:43.019748 12772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:43.019865 12772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:43.019975 12772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:29:43.025923 12772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:34995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:43.027199 12772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:43.028380 12772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:43.038970 12780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:43.038980 12779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:43.039191 12772 server_base.cc:1034] running on GCE node
W20260811 22:29:43.039320 12782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:43.039752 12772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:43.040618 12772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:43.041803 12772 hybrid_clock.cc:648] HybridClock initialized: now 1786487383041762 us; error 58 us; skew 500 ppm
Aug 11 22:29:43 dist-test-slave-mvvj krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487383, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:43.044927 12772 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:29:43.045910 12772 webserver.cc:460] Webserver started at http://127.24.71.194:34249/ using document root <none> and password file <none>
I20260811 22:29:43.046599 12772 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:43.046702 12772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:43.046957 12772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:43.049401 12772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "0b2179199b9247aa984444462cd1920c"
format_stamp: "Formatted at 2026-08-11 22:29:43 on dist-test-slave-mvvj"
server_key: "cded56797a635a4794885246f21df6b5"
server_key_iv: "b00faa315c930cacc6d903c1268bed3f"
server_key_version: "encryptionkey@0"
I20260811 22:29:43.050128 12772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "0b2179199b9247aa984444462cd1920c"
format_stamp: "Formatted at 2026-08-11 22:29:43 on dist-test-slave-mvvj"
server_key: "cded56797a635a4794885246f21df6b5"
server_key_iv: "b00faa315c930cacc6d903c1268bed3f"
server_key_version: "encryptionkey@0"
I20260811 22:29:43.054262 12772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:29:43.057053 12789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:43.058554 12772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:43.058708 12772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "0b2179199b9247aa984444462cd1920c"
format_stamp: "Formatted at 2026-08-11 22:29:43 on dist-test-slave-mvvj"
server_key: "cded56797a635a4794885246f21df6b5"
server_key_iv: "b00faa315c930cacc6d903c1268bed3f"
server_key_version: "encryptionkey@0"
I20260811 22:29:43.058837 12772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:43.081041 12772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:43.084615 12772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:43.084838 12772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:43.085485 12772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:43.086408 12772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:43.086483 12772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:43.086635 12772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:43.086694 12772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:43.097216 12772 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:46811
I20260811 22:29:43.097239 12919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:46811 every 8 connection(s)
I20260811 22:29:43.098197 12772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:29:43.108098 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 12772
I20260811 22:29:43.108228 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:29:43.108539 24863 external_mini_cluster.cc:1428] Setting key e7c77c535049706dbea2786cd837dc9f
Aug 11 22:29:43 dist-test-slave-mvvj krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487383, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:43.113961 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.100283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:43465 (local address 127.24.71.254:34995)
0811 22:29:43.100451 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.100456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.101359 (+ 903us) server_negotiation.cc:407] Connection header received
0811 22:29:43.102794 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.102797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.102847 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.102927 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:43.104254 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.104825 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.105988 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.106210 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.106322 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.109391 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:43.109394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:43.109399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:43.109433 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:43.111365 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:43.112020 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:43.112026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:43.112031 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:43.112137 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:43.112504 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:43.112510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:43.112515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:43.112800 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:43.112915 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.113696 (+ 781us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.113823 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:29:43.115460 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.100712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:34995 (local address 127.24.71.194:43465)
0811 22:29:43.101198 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:43.101244 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:43.102576 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:43.103103 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:43.103110 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:43.103527 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:43.104109 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.104125 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.104960 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.104963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.105865 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.105875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.106370 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.106381 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.106549 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.107263 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:43.107281 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:43.109203 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:43.111516 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:43.111523 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:43.111546 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:43.111913 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:43.112241 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:43.112243 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:43.112244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:43.112397 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:43.112925 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:43.112935 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:43.113371 (+ 436us) client_negotiation.cc:769] Sending connection context
0811 22:29:43.113611 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 22:29:43.114249 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":115,"threads_started":1}
I20260811 22:29:43.117014 12920 heartbeater.cc:347] Connected to a master server at 127.24.71.254:34995
I20260811 22:29:43.117338 12920 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:43.117945 12920 heartbeater.cc:511] Master 127.24.71.254:34995 requested a full tablet report, sending...
I20260811 22:29:43.119251 12554 ts_manager.cc:194] Registered new tserver with Master: 0b2179199b9247aa984444462cd1920c (127.24.71.194:46811)
I20260811 22:29:43.120041 12554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:43465
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:29:43.144399 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:34995
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:43.266106 12925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:43.266417 12925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:43.266462 12925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:43.270787 12925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:43.270911 12925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:43.271020 12925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:29:43.276741 12925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:34995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:43.278335 12925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:43.279587 12925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:43.290374 12930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:43.290478 12931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:43.290545 12925 server_base.cc:1034] running on GCE node
W20260811 22:29:43.290374 12933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:43.291160 12925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:43.291929 12925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:43.293131 12925 hybrid_clock.cc:648] HybridClock initialized: now 1786487383293060 us; error 83 us; skew 500 ppm
Aug 11 22:29:43 dist-test-slave-mvvj krb5kdc[12503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487383, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:43.296342 12925 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:29:43.297375 12925 webserver.cc:460] Webserver started at http://127.24.71.195:42571/ using document root <none> and password file <none>
I20260811 22:29:43.297983 12925 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:43.298081 12925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:43.298341 12925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:43.300580 12925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "7e3a74fdae1b443d9d8492a753d2bd2c"
format_stamp: "Formatted at 2026-08-11 22:29:43 on dist-test-slave-mvvj"
server_key: "a567337e1054a6d401666cf9ed18e386"
server_key_iv: "891ced08ec8b8f0b632e879850a71791"
server_key_version: "encryptionkey@0"
I20260811 22:29:43.301179 12925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "7e3a74fdae1b443d9d8492a753d2bd2c"
format_stamp: "Formatted at 2026-08-11 22:29:43 on dist-test-slave-mvvj"
server_key: "a567337e1054a6d401666cf9ed18e386"
server_key_iv: "891ced08ec8b8f0b632e879850a71791"
server_key_version: "encryptionkey@0"
I20260811 22:29:43.305435 12925 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:29:43.308116 12942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:43.309437 12925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:29:43.309587 12925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "7e3a74fdae1b443d9d8492a753d2bd2c"
format_stamp: "Formatted at 2026-08-11 22:29:43 on dist-test-slave-mvvj"
server_key: "a567337e1054a6d401666cf9ed18e386"
server_key_iv: "891ced08ec8b8f0b632e879850a71791"
server_key_version: "encryptionkey@0"
I20260811 22:29:43.309711 12925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:43.322388 12925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:43.326052 12925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:43.326280 12925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:43.326939 12925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:43.327911 12925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:43.327965 12925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:43.328032 12925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:43.328069 12925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:43.337733 12925 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:33513
I20260811 22:29:43.337751 13071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:33513 every 8 connection(s)
I20260811 22:29:43.338768 12925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:29:43.343645 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 12925
I20260811 22:29:43.343825 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:29:43.344213 24863 external_mini_cluster.cc:1428] Setting key 8f4d19543a7e8cfe2b4c46d3c732c9ac
Aug 11 22:29:43 dist-test-slave-mvvj krb5kdc[12503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487383, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:43.355135 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.341078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:60199 (local address 127.24.71.254:34995)
0811 22:29:43.341264 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.341268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.342246 (+ 978us) server_negotiation.cc:407] Connection header received
0811 22:29:43.343688 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.343691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.343756 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.343905 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:43.345409 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.346005 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.347342 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.347590 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.347716 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.350989 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:43.350993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:43.350995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:43.351025 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:43.352717 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:43.353342 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:43.353351 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:43.353353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:43.353443 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:43.353838 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:43.353846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:43.353848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:43.354084 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:43.354209 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.354801 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.354951 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:29:43.356164 13074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.341485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:34995 (local address 127.24.71.195:60199)
0811 22:29:43.342066 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:43.342113 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:43.343442 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:43.344135 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:43.344145 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:43.344678 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:43.345231 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.345247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.346152 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.346158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.347181 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.347192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.347762 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.347767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.348027 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.348761 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:43.348779 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:43.350748 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:43.352873 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:43.352880 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:43.352900 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:43.353204 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:43.353553 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:43.353557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:43.353560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:43.353725 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:43.354209 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:43.354219 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:43.354530 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 22:29:43.354754 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 22:29:43.355148 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":158,"threads_started":1}
I20260811 22:29:43.357797 13072 heartbeater.cc:347] Connected to a master server at 127.24.71.254:34995
I20260811 22:29:43.358106 13072 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:43.358789 13072 heartbeater.cc:511] Master 127.24.71.254:34995 requested a full tablet report, sending...
I20260811 22:29:43.360041 12554 ts_manager.cc:194] Registered new tserver with Master: 7e3a74fdae1b443d9d8492a753d2bd2c (127.24.71.195:33513)
I20260811 22:29:43.360700 12554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:60199
I20260811 22:29:43.371658 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:29:43.384014 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.374148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42926 (local address 127.24.71.254:34995)
0811 22:29:43.374437 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.374444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.374549 (+ 105us) server_negotiation.cc:407] Connection header received
0811 22:29:43.374759 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.374763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.374812 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.374927 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:43.375736 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.376477 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.377674 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.377938 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.378067 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.380043 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:43.380047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:43.380051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:43.380088 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:43.381733 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:43.382422 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:43.382426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:43.382430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:43.382523 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:43.382900 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:43.382903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:43.382905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:43.383235 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:43.383412 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.383701 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.383810 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 22:29:43.387960 12554 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:29:43.391469 12554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:29:43.407799 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.401668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:46811 (local address 127.0.0.1:43972)
0811 22:29:43.402109 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:43.402171 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:43.402317 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:43.402940 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:43.402948 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:43.402972 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:43.403126 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.403137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.404594 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.404597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.405950 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.405961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.407173 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.407177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.407292 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.407333 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:29:43.407488 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 22:29:43.407629 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
I20260811 22:29:43.407814 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.401718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33513 (local address 127.0.0.1:45114)
0811 22:29:43.402617 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:43.402632 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:43.402727 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:43.404035 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:43.404038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:43.404057 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:43.404158 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.404163 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.405322 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.405326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.406334 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.406345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.407475 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.407482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.407573 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.407595 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:29:43.407668 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:29:43.407725 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":89,"threads_started":1}
I20260811 22:29:43.408408 13086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.401778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.24.71.194:46811)
0811 22:29:43.402256 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.402269 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.402294 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:29:43.402539 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.402545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.402770 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.402949 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:43.403303 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.404460 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.406143 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.407034 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.407160 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.407320 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.408153 (+ 833us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.408270 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":141,"threads_started":1}
I20260811 22:29:43.408458 13088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.402602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.24.71.195:33513)
0811 22:29:43.403406 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.403419 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.403441 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:29:43.403513 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.403521 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.403702 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.403867 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:43.404307 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.405167 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.406478 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.407340 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.407449 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.407589 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.407702 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.407805 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":113,"threads_started":1}
I20260811 22:29:43.408465 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.402479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37015 (local address 127.0.0.1:46400)
0811 22:29:43.402810 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:43.402822 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:43.402900 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:43.404291 (+ 1391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:43.404294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:43.404315 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:43.404411 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.404418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.405802 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.405806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.406919 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.406929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.408075 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.408080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.408173 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.408197 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:29:43.408261 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 22:29:43.408325 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":110,"threads_started":1}
I20260811 22:29:43.408560 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.403100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.24.71.193:37015)
0811 22:29:43.403876 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.403886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.403909 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:29:43.403990 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.403996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.404148 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.404310 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:43.404544 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.405670 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.407073 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.407947 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.408073 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.408261 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.408310 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.408407 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":103,"threads_started":1}
I20260811 22:29:43.411883 12996 tablet_service.cc:1467] Processing CreateTablet for tablet 025d1a294b5d4d4aa0c314d0a9806ed4 (DEFAULT_TABLE table=test-table [id=a3a039a4d53047a4a62a559194848aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:43.411880 12695 tablet_service.cc:1467] Processing CreateTablet for tablet 025d1a294b5d4d4aa0c314d0a9806ed4 (DEFAULT_TABLE table=test-table [id=a3a039a4d53047a4a62a559194848aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:43.411880 12843 tablet_service.cc:1467] Processing CreateTablet for tablet 025d1a294b5d4d4aa0c314d0a9806ed4 (DEFAULT_TABLE table=test-table [id=a3a039a4d53047a4a62a559194848aef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:43.413041 12996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 025d1a294b5d4d4aa0c314d0a9806ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:43.413044 12695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 025d1a294b5d4d4aa0c314d0a9806ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:43.413043 12843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 025d1a294b5d4d4aa0c314d0a9806ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:43.420261 13090 tablet_bootstrap.cc:492] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: Bootstrap starting.
I20260811 22:29:43.421418 13091 tablet_bootstrap.cc:492] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d: Bootstrap starting.
I20260811 22:29:43.421657 13094 tablet_bootstrap.cc:492] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: Bootstrap starting.
I20260811 22:29:43.422791 13090 tablet_bootstrap.cc:654] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:43.423755 13091 tablet_bootstrap.cc:654] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:43.423748 13094 tablet_bootstrap.cc:654] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:43.424067 13090 log.cc:824] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:43.424701 13091 log.cc:824] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:43.425057 13094 log.cc:824] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:43.426570 13090 tablet_bootstrap.cc:492] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: No bootstrap required, opened a new log
I20260811 22:29:43.426781 13091 tablet_bootstrap.cc:492] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d: No bootstrap required, opened a new log
I20260811 22:29:43.426817 13090 ts_tablet_manager.cc:1397] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:29:43.426934 13094 tablet_bootstrap.cc:492] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: No bootstrap required, opened a new log
I20260811 22:29:43.426985 13091 ts_tablet_manager.cc:1397] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:29:43.427106 13094 ts_tablet_manager.cc:1397] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:29:43.430083 13090 raft_consensus.cc:348] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } }
I20260811 22:29:43.430368 13090 raft_consensus.cc:374] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:43.430435 13090 raft_consensus.cc:729] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e3a74fdae1b443d9d8492a753d2bd2c, State: Initialized, Role: FOLLOWER
I20260811 22:29:43.430925 13090 consensus_queue.cc:260] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } }
I20260811 22:29:43.431187 13094 raft_consensus.cc:348] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } }
I20260811 22:29:43.431461 13094 raft_consensus.cc:374] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:43.431543 13094 raft_consensus.cc:729] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2179199b9247aa984444462cd1920c, State: Initialized, Role: FOLLOWER
I20260811 22:29:43.431586 13091 raft_consensus.cc:348] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } }
I20260811 22:29:43.431869 13091 raft_consensus.cc:374] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:43.431877 13072 heartbeater.cc:503] Master 127.24.71.254:34995 was elected leader, sending a full tablet report...
I20260811 22:29:43.431944 13091 raft_consensus.cc:729] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6f1d179eff74ee7a40366b26e220f9d, State: Initialized, Role: FOLLOWER
I20260811 22:29:43.432181 13090 ts_tablet_manager.cc:1428] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 22:29:43.432466 13091 consensus_queue.cc:260] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } }
I20260811 22:29:43.433353 12766 heartbeater.cc:503] Master 127.24.71.254:34995 was elected leader, sending a full tablet report...
I20260811 22:29:43.433574 13091 ts_tablet_manager.cc:1428] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:29:43.433666 13094 consensus_queue.cc:260] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } }
I20260811 22:29:43.434741 12920 heartbeater.cc:503] Master 127.24.71.254:34995 was elected leader, sending a full tablet report...
I20260811 22:29:43.434732 13094 ts_tablet_manager.cc:1428] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260811 22:29:43.568423 13100 raft_consensus.cc:482] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:43.568759 13100 raft_consensus.cc:504] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } }
I20260811 22:29:43.570017 13100 leader_election.cc:288] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e3a74fdae1b443d9d8492a753d2bd2c (127.24.71.195:33513), d6f1d179eff74ee7a40366b26e220f9d (127.24.71.193:37015)
I20260811 22:29:43.575477 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.570870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:59519 (local address 127.24.71.193:37015)
0811 22:29:43.571030 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.571034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.571146 (+ 112us) server_negotiation.cc:407] Connection header received
0811 22:29:43.571227 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.571229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.571271 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.571378 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:43.571681 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.572419 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.574104 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.574751 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.574857 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.574916 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.575259 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.575334 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:29:43.575495 13088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.570568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:60949 (local address 127.24.71.195:33513)
0811 22:29:43.570705 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.570713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.570732 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:29:43.570803 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.570806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.570864 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.570964 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:43.571341 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.572159 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.573994 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.574647 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.574783 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.574857 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.575259 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.575334 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:29:43.575510 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.570424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:33513 (local address 127.24.71.194:60949)
0811 22:29:43.570565 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:43.570582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:43.570686 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:43.571051 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:43.571055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:43.571081 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:43.571216 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.571226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.572286 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.572289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.573839 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.573853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.574791 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.574795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.575048 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.575142 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 22:29:43.575230 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:29:43.575344 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":146}
I20260811 22:29:43.575518 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.570705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37015 (local address 127.24.71.194:59519)
0811 22:29:43.571036 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:43.571049 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:43.571144 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:43.571442 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:43.571444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:43.571460 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:43.571557 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.571561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.572512 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.572515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.573963 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:43.573974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.574865 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.574868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:43.575021 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.575136 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 22:29:43.575230 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:29:43.575384 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":72,"thread_start_us":147,"threads_started":1}
I20260811 22:29:43.576411 13020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "025d1a294b5d4d4aa0c314d0a9806ed4" candidate_uuid: "0b2179199b9247aa984444462cd1920c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" is_pre_election: true
I20260811 22:29:43.576604 12717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "025d1a294b5d4d4aa0c314d0a9806ed4" candidate_uuid: "0b2179199b9247aa984444462cd1920c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6f1d179eff74ee7a40366b26e220f9d" is_pre_election: true
I20260811 22:29:43.576880 13020 raft_consensus.cc:2454] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2179199b9247aa984444462cd1920c in term 0.
I20260811 22:29:43.576989 12717 raft_consensus.cc:2454] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2179199b9247aa984444462cd1920c in term 0.
I20260811 22:29:43.577487 12793 leader_election.cc:302] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2179199b9247aa984444462cd1920c, 7e3a74fdae1b443d9d8492a753d2bd2c; no voters:
I20260811 22:29:43.577796 13100 raft_consensus.cc:2781] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:29:43.577920 13100 raft_consensus.cc:482] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:29:43.577965 13100 raft_consensus.cc:3037] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:43.579085 13100 raft_consensus.cc:504] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } }
I20260811 22:29:43.579459 13100 leader_election.cc:288] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [CANDIDATE]: Term 1 election: Requested vote from peers 7e3a74fdae1b443d9d8492a753d2bd2c (127.24.71.195:33513), d6f1d179eff74ee7a40366b26e220f9d (127.24.71.193:37015)
I20260811 22:29:43.579830 13020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "025d1a294b5d4d4aa0c314d0a9806ed4" candidate_uuid: "0b2179199b9247aa984444462cd1920c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c"
I20260811 22:29:43.579909 12717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "025d1a294b5d4d4aa0c314d0a9806ed4" candidate_uuid: "0b2179199b9247aa984444462cd1920c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6f1d179eff74ee7a40366b26e220f9d"
I20260811 22:29:43.579999 13020 raft_consensus.cc:3037] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:43.580017 12717 raft_consensus.cc:3037] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:43.581369 12717 raft_consensus.cc:2454] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2179199b9247aa984444462cd1920c in term 1.
I20260811 22:29:43.581369 13020 raft_consensus.cc:2454] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2179199b9247aa984444462cd1920c in term 1.
I20260811 22:29:43.581712 12793 leader_election.cc:302] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2179199b9247aa984444462cd1920c, d6f1d179eff74ee7a40366b26e220f9d; no voters:
I20260811 22:29:43.581907 13100 raft_consensus.cc:2781] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:43.582151 13100 raft_consensus.cc:686] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 1 LEADER]: Becoming Leader. State: Replica: 0b2179199b9247aa984444462cd1920c, State: Running, Role: LEADER
I20260811 22:29:43.582564 13100 consensus_queue.cc:237] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } }
I20260811 22:29:43.586076 12553 catalog_manager.cc:5495] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b2179199b9247aa984444462cd1920c (127.24.71.194). New cstate: current_term: 1 leader_uuid: "0b2179199b9247aa984444462cd1920c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b2179199b9247aa984444462cd1920c" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 46811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 } health_report { overall_health: UNKNOWN } } }
W20260811 22:29:43.589834 13073 tablet.cc:2369] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:29:43.593429 12767 tablet.cc:2369] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:29:43.599431 12921 tablet.cc:2369] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:29:43.603574 13086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.599792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.24.71.194:46811)
0811 22:29:43.599963 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.599968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.600013 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:29:43.600091 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.600095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.600163 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.600248 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:29:43.600561 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.601077 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.602022 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.602242 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.602332 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.602696 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:29:43.603100 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:29:43.603212 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.603365 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.603404 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:29:43.610023 12717 raft_consensus.cc:1261] T 025d1a294b5d4d4aa0c314d0a9806ed4 P d6f1d179eff74ee7a40366b26e220f9d [term 1 FOLLOWER]: Refusing update from remote peer 0b2179199b9247aa984444462cd1920c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:43.610037 13020 raft_consensus.cc:1261] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [term 1 FOLLOWER]: Refusing update from remote peer 0b2179199b9247aa984444462cd1920c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:43.610914 13100 consensus_queue.cc:1035] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e3a74fdae1b443d9d8492a753d2bd2c" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 33513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:43.611176 13102 consensus_queue.cc:1035] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f1d179eff74ee7a40366b26e220f9d" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 37015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:43.621842 13105 mvcc.cc:205] Tried to move back new op lower bound from 7317452323258159104 to 7317452323157045248. Current Snapshot: MvccSnapshot[applied={T|T < 7317452323258159104 or (T in {7317452323258159104})}]
I20260811 22:29:43.629061 12553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:43.629271 12553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:43.632035 12553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b24c8eb91c844c0a32cf90ad1af8836: Sending DeleteTablet for 3 replicas of tablet 025d1a294b5d4d4aa0c314d0a9806ed4
I20260811 22:29:43.632933 12843 tablet_service.cc:1514] Processing DeleteTablet for tablet 025d1a294b5d4d4aa0c314d0a9806ed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:43 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:43972
I20260811 22:29:43.633055 12996 tablet_service.cc:1514] Processing DeleteTablet for tablet 025d1a294b5d4d4aa0c314d0a9806ed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:43 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:45114
I20260811 22:29:43.633760 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 12626
I20260811 22:29:43.633834 13114 tablet_replica.cc:330] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: stopping tablet replica
I20260811 22:29:43.633827 12695 tablet_service.cc:1514] Processing DeleteTablet for tablet 025d1a294b5d4d4aa0c314d0a9806ed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:43 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:46400
I20260811 22:29:43.634207 13114 raft_consensus.cc:2229] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:29:43.634604 13114 raft_consensus.cc:2258] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:43.633651 13115 tablet_replica.cc:330] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: stopping tablet replica
I20260811 22:29:43.635668 13115 raft_consensus.cc:2229] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:43.636052 13115 raft_consensus.cc:2258] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:43.638160 13115 ts_tablet_manager.cc:1905] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:43.641026 13115 ts_tablet_manager.cc:1918] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:43.641181 13115 log.cc:1196] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/025d1a294b5d4d4aa0c314d0a9806ed4
I20260811 22:29:43.641108 13114 ts_tablet_manager.cc:1905] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:43.641561 13115 ts_tablet_manager.cc:1939] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 7e3a74fdae1b443d9d8492a753d2bd2c: Deleting consensus metadata
I20260811 22:29:43.643071 12541 catalog_manager.cc:4841] TS 7e3a74fdae1b443d9d8492a753d2bd2c (127.24.71.195:33513): tablet 025d1a294b5d4d4aa0c314d0a9806ed4 (table test-table [id=a3a039a4d53047a4a62a559194848aef]) successfully deleted
I20260811 22:29:43.644253 13114 ts_tablet_manager.cc:1918] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:43.644387 13114 log.cc:1196] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/025d1a294b5d4d4aa0c314d0a9806ed4
I20260811 22:29:43.644673 13114 ts_tablet_manager.cc:1939] T 025d1a294b5d4d4aa0c314d0a9806ed4 P 0b2179199b9247aa984444462cd1920c: Deleting consensus metadata
I20260811 22:29:43.645838 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 12772
W20260811 22:29:43.645834 12541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:37015 (error 108)
I20260811 22:29:43.645877 12539 catalog_manager.cc:4841] TS 0b2179199b9247aa984444462cd1920c (127.24.71.194:46811): tablet 025d1a294b5d4d4aa0c314d0a9806ed4 (table test-table [id=a3a039a4d53047a4a62a559194848aef]) successfully deleted
I20260811 22:29:43.650089 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.649716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:37015 (local address 127.0.0.1:46412)
0811 22:29:43.649868 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:43.649967 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:29:43.650352 12541 catalog_manager.cc:4584] TS d6f1d179eff74ee7a40366b26e220f9d (127.24.71.193:37015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 025d1a294b5d4d4aa0c314d0a9806ed4: Network error: Client connection negotiation failed: client connection to 127.24.71.193:37015: connect: Connection refused (error 111)
I20260811 22:29:43.655165 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 12925
I20260811 22:29:43.664609 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 12518
2026-08-11T22:29:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2650 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T22:29:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:43Z Disabled control of system clock
I20260811 22:29:43.701553 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:32923
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:41425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:32923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:29:43.823740 13120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:43.824082 13120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:43.824121 13120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:43.828138 13120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:43.828207 13120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:43.828222 13120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:43.828240 13120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:43.828255 13120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:43.833168 13120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:41425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:32923
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:32923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:43.834364 13120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:43.836054 13120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:43.845810 13129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:43.845796 13126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:43.845799 13127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:43.845868 13120 server_base.cc:1034] running on GCE node
I20260811 22:29:43.846843 13120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:43.848304 13120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:43.849525 13120 hybrid_clock.cc:648] HybridClock initialized: now 1786487383849509 us; error 35 us; skew 500 ppm
I20260811 22:29:43.851913 13120 webserver.cc:460] Webserver started at http://127.24.71.254:45903/ using document root <none> and password file <none>
I20260811 22:29:43.852629 13120 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:43.852749 13120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:43.853029 13120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:43.855422 13120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "b9a00e0a90514fc792eb6a0185d7735f"
format_stamp: "Formatted at 2026-08-11 22:29:43 on dist-test-slave-mvvj"
server_key: "494cdc9ebfdcd4bd1e6e88d1b1559735"
server_key_iv: "e547ed91e249c6b8878dea717163aa1f"
server_key_version: "encryptionkey@0"
I20260811 22:29:43.856195 13120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "b9a00e0a90514fc792eb6a0185d7735f"
format_stamp: "Formatted at 2026-08-11 22:29:43 on dist-test-slave-mvvj"
server_key: "494cdc9ebfdcd4bd1e6e88d1b1559735"
server_key_iv: "e547ed91e249c6b8878dea717163aa1f"
server_key_version: "encryptionkey@0"
I20260811 22:29:43.860615 13120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:29:43.863620 13135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:43.865258 13120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:29:43.865463 13120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "b9a00e0a90514fc792eb6a0185d7735f"
format_stamp: "Formatted at 2026-08-11 22:29:43 on dist-test-slave-mvvj"
server_key: "494cdc9ebfdcd4bd1e6e88d1b1559735"
server_key_iv: "e547ed91e249c6b8878dea717163aa1f"
server_key_version: "encryptionkey@0"
I20260811 22:29:43.865618 13120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:43.889364 13120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:43.890225 13120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:43.890462 13120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:43.898257 13120 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:32923
I20260811 22:29:43.898262 13193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:32923 every 8 connection(s)
I20260811 22:29:43.899313 13120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:43.902681 13195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:43.908537 13195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f: Bootstrap starting.
I20260811 22:29:43.909457 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 13120
I20260811 22:29:43.909590 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:43.909927 24863 external_mini_cluster.cc:1428] Setting key 6366f6b495f6fe973444a2fb9b7fbd1f
I20260811 22:29:43.912034 13195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:43.912878 13195 log.cc:824] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:43.915288 13195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f: No bootstrap required, opened a new log
I20260811 22:29:43.919636 13195 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a00e0a90514fc792eb6a0185d7735f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 32923 } }
I20260811 22:29:43.919974 13195 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:43.920069 13195 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a00e0a90514fc792eb6a0185d7735f, State: Initialized, Role: FOLLOWER
I20260811 22:29:43.920609 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:43.911358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.24.71.254:32923)
0811 22:29:43.912260 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:43.912278 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:43.912320 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:29:43.913156 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:43.913178 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:43.913629 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:43.914009 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:43.914491 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.915539 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.916572 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:43.917089 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:43.917291 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:43.918367 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:43.918374 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:43.918396 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:43.918855 (+ 459us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:43.918922 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:43.918947 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:43.919073 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:43.919247 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:29:43.919554 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":149,"threads_started":1}
I20260811 22:29:43.920626 13195 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a00e0a90514fc792eb6a0185d7735f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 32923 } }
I20260811 22:29:43.920868 13195 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:43.920941 13195 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:43.921078 13195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:43.922454 13195 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a00e0a90514fc792eb6a0185d7735f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 32923 } }
I20260811 22:29:43.922863 13195 leader_election.cc:302] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9a00e0a90514fc792eb6a0185d7735f; no voters:
I20260811 22:29:43.923287 13195 leader_election.cc:288] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:43.923532 13200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:43.923784 13200 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [term 1 LEADER]: Becoming Leader. State: Replica: b9a00e0a90514fc792eb6a0185d7735f, State: Running, Role: LEADER
I20260811 22:29:43.924134 13200 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a00e0a90514fc792eb6a0185d7735f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 32923 } }
I20260811 22:29:43.924793 13195 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:43.926000 13201 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9a00e0a90514fc792eb6a0185d7735f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a00e0a90514fc792eb6a0185d7735f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 32923 } } }
I20260811 22:29:43.926173 13201 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:43.926016 13202 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9a00e0a90514fc792eb6a0185d7735f. Latest consensus state: current_term: 1 leader_uuid: "b9a00e0a90514fc792eb6a0185d7735f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a00e0a90514fc792eb6a0185d7735f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 32923 } } }
I20260811 22:29:43.926520 13202 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:43.926592 13210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:43.930100 13210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:43.937034 13210 catalog_manager.cc:1347] Generated new cluster ID: 54b91b1551174d009627263b9590a3d0
I20260811 22:29:43.937136 13210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:43.946959 13210 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:43.948343 13210 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:43.962306 13210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f: Generated new TSK 0
I20260811 22:29:43.963276 13210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:29:43.975301 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:32923
--builtin_ntp_servers=127.24.71.212:41425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:29:44.104748 13220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:44.105041 13220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:44.105080 13220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:44.109315 13220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:44.109390 13220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:44.109448 13220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:29:44.114454 13220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:41425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:32923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:44.115768 13220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:44.117245 13220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:29:44.127807 13220 server_base.cc:1034] running on GCE node
W20260811 22:29:44.127820 13226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:44.127820 13227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:44.127817 13229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:44.128687 13220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:44.129382 13220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:44.130607 13220 hybrid_clock.cc:648] HybridClock initialized: now 1786487384130573 us; error 46 us; skew 500 ppm
I20260811 22:29:44.132622 13220 webserver.cc:460] Webserver started at http://127.24.71.193:34567/ using document root <none> and password file <none>
I20260811 22:29:44.133241 13220 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:44.133339 13220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:44.133638 13220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:44.135936 13220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "2590b1fd03304e21afe4b4a403b46fb2"
format_stamp: "Formatted at 2026-08-11 22:29:44 on dist-test-slave-mvvj"
server_key: "a7f57d26fe8efe94a84a1c92ac7ced8c"
server_key_iv: "cf708a1221b1b4a91bce03903f0728dc"
server_key_version: "encryptionkey@0"
I20260811 22:29:44.136574 13220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "2590b1fd03304e21afe4b4a403b46fb2"
format_stamp: "Formatted at 2026-08-11 22:29:44 on dist-test-slave-mvvj"
server_key: "a7f57d26fe8efe94a84a1c92ac7ced8c"
server_key_iv: "cf708a1221b1b4a91bce03903f0728dc"
server_key_version: "encryptionkey@0"
I20260811 22:29:44.140640 13220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:29:44.143383 13237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:44.144862 13220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:29:44.145011 13220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "2590b1fd03304e21afe4b4a403b46fb2"
format_stamp: "Formatted at 2026-08-11 22:29:44 on dist-test-slave-mvvj"
server_key: "a7f57d26fe8efe94a84a1c92ac7ced8c"
server_key_iv: "cf708a1221b1b4a91bce03903f0728dc"
server_key_version: "encryptionkey@0"
I20260811 22:29:44.145140 13220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:44.169785 13220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:44.170665 13220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:44.170917 13220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:44.171586 13220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:44.172638 13220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:44.172690 13220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:44.172732 13220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:44.172784 13220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:44.182358 13220 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:46749
I20260811 22:29:44.182521 13367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:46749 every 8 connection(s)
I20260811 22:29:44.183409 13220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:29:44.184664 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 13220
I20260811 22:29:44.184799 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:29:44.185178 24863 external_mini_cluster.cc:1428] Setting key 8ddf570cd4a4d4be826036b88656c7a6
I20260811 22:29:44.187641 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:32923
--builtin_ntp_servers=127.24.71.212:41425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:29:44.196115 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.185737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:51569 (local address 127.24.71.254:32923)
0811 22:29:44.185983 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.185989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.187383 (+ 1394us) server_negotiation.cc:407] Connection header received
0811 22:29:44.189662 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.189667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.189745 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.189856 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:44.190968 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.191587 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.193614 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.193852 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.193943 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.195515 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:44.195518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:44.195520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:44.195537 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:44.195558 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:44.195566 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:44.195630 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:44.195908 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 22:29:44.195986 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260811 22:29:44.197739 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.186303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:32923 (local address 127.24.71.193:51569)
0811 22:29:44.187167 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:44.187233 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:44.189341 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:44.190126 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:44.190134 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:44.190232 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:44.190784 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.190798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.191743 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.191750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.193452 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.193471 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.194055 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.194062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.194330 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.195115 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:29:44.195118 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:44.195190 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:29:44.195303 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:44.195667 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:44.195681 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:44.195690 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:29:44.195900 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 22:29:44.196303 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":136,"threads_started":1}
I20260811 22:29:44.199450 13368 heartbeater.cc:347] Connected to a master server at 127.24.71.254:32923
I20260811 22:29:44.199755 13368 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:44.200459 13368 heartbeater.cc:511] Master 127.24.71.254:32923 requested a full tablet report, sending...
I20260811 22:29:44.202385 13154 ts_manager.cc:194] Registered new tserver with Master: 2590b1fd03304e21afe4b4a403b46fb2 (127.24.71.193:46749)
I20260811 22:29:44.204092 13154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.193:51569
W20260811 22:29:44.310364 13371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:44.310667 13371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:44.310709 13371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:44.314945 13371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:44.315035 13371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:44.315096 13371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:29:44.320145 13371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:41425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:32923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:44.321367 13371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:44.322811 13371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:44.334156 13377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:44.334216 13371 server_base.cc:1034] running on GCE node
W20260811 22:29:44.334132 13376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:44.334105 13379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:44.334791 13371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:44.335559 13371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:44.336778 13371 hybrid_clock.cc:648] HybridClock initialized: now 1786487384336733 us; error 65 us; skew 500 ppm
I20260811 22:29:44.338864 13371 webserver.cc:460] Webserver started at http://127.24.71.194:41407/ using document root <none> and password file <none>
I20260811 22:29:44.339473 13371 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:44.339538 13371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:44.339717 13371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:44.341939 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "dd9e0b8d45f1419f86ac25002248e9f9"
format_stamp: "Formatted at 2026-08-11 22:29:44 on dist-test-slave-mvvj"
server_key: "d9685a82b7d34e3e568bc3100528dfe3"
server_key_iv: "e8f1434f152de7a17add91ea37220d87"
server_key_version: "encryptionkey@0"
I20260811 22:29:44.342517 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "dd9e0b8d45f1419f86ac25002248e9f9"
format_stamp: "Formatted at 2026-08-11 22:29:44 on dist-test-slave-mvvj"
server_key: "d9685a82b7d34e3e568bc3100528dfe3"
server_key_iv: "e8f1434f152de7a17add91ea37220d87"
server_key_version: "encryptionkey@0"
I20260811 22:29:44.346637 13371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:29:44.349676 13388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:44.351197 13371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:29:44.351351 13371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "dd9e0b8d45f1419f86ac25002248e9f9"
format_stamp: "Formatted at 2026-08-11 22:29:44 on dist-test-slave-mvvj"
server_key: "d9685a82b7d34e3e568bc3100528dfe3"
server_key_iv: "e8f1434f152de7a17add91ea37220d87"
server_key_version: "encryptionkey@0"
I20260811 22:29:44.351500 13371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:44.362566 13371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:44.363389 13371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:44.363626 13371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:44.364344 13371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:44.365372 13371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:44.365432 13371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:44.365499 13371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:44.365532 13371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:44.376156 13371 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:41235
I20260811 22:29:44.376204 13511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:41235 every 8 connection(s)
I20260811 22:29:44.377193 13371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:29:44.386759 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 13371
I20260811 22:29:44.386893 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:29:44.387312 24863 external_mini_cluster.cc:1428] Setting key f34270a89df964147ca1e93a2f02f5c9
I20260811 22:29:44.387809 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.379263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:39085 (local address 127.24.71.254:32923)
0811 22:29:44.379412 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.379416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.380666 (+ 1250us) server_negotiation.cc:407] Connection header received
0811 22:29:44.382134 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.382137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.382191 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.382280 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:44.383359 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.384055 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.385307 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.385583 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.385761 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.387186 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:44.387191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:44.387195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:44.387212 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:44.387239 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:44.387251 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:44.387336 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:44.387584 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 22:29:44.387658 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:29:44.389024 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.379667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:32923 (local address 127.24.71.194:39085)
0811 22:29:44.380495 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:44.380550 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:44.381895 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:44.382460 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:44.382520 (+ 60us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:44.382613 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:44.383132 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.383145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.384192 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.384195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.385166 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.385176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.385838 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.385844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.386051 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.386777 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:29:44.386778 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:44.386857 (+ 79us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:29:44.386991 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:44.387347 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:44.387378 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:44.387387 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:29:44.387573 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 22:29:44.387918 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":218,"threads_started":1}
I20260811 22:29:44.389794 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:32923
--builtin_ntp_servers=127.24.71.212:41425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:29:44.390493 13512 heartbeater.cc:347] Connected to a master server at 127.24.71.254:32923
I20260811 22:29:44.390767 13512 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:44.391371 13512 heartbeater.cc:511] Master 127.24.71.254:32923 requested a full tablet report, sending...
I20260811 22:29:44.392797 13154 ts_manager.cc:194] Registered new tserver with Master: dd9e0b8d45f1419f86ac25002248e9f9 (127.24.71.194:41235)
I20260811 22:29:44.393571 13154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.194:39085
W20260811 22:29:44.515890 13515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:44.516203 13515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:44.516242 13515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:44.520339 13515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:44.520399 13515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:44.520455 13515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:29:44.525418 13515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:41425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:32923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:44.526652 13515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:44.528164 13515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:44.538365 13522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:44.538370 13525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:44.538337 13515 server_base.cc:1034] running on GCE node
W20260811 22:29:44.538437 13523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:44.539167 13515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:44.539906 13515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:44.541110 13515 hybrid_clock.cc:648] HybridClock initialized: now 1786487384541091 us; error 34 us; skew 500 ppm
I20260811 22:29:44.543130 13515 webserver.cc:460] Webserver started at http://127.24.71.195:46133/ using document root <none> and password file <none>
I20260811 22:29:44.543733 13515 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:44.543833 13515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:44.544111 13515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:44.546219 13515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "e3fc32d9ad86432e873fb64ecf01f237"
format_stamp: "Formatted at 2026-08-11 22:29:44 on dist-test-slave-mvvj"
server_key: "cb8bd2aaf881d2cd6bc46152fdc27602"
server_key_iv: "cf40c262f59eb4c8c086a188ff1c411c"
server_key_version: "encryptionkey@0"
I20260811 22:29:44.546797 13515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "e3fc32d9ad86432e873fb64ecf01f237"
format_stamp: "Formatted at 2026-08-11 22:29:44 on dist-test-slave-mvvj"
server_key: "cb8bd2aaf881d2cd6bc46152fdc27602"
server_key_iv: "cf40c262f59eb4c8c086a188ff1c411c"
server_key_version: "encryptionkey@0"
I20260811 22:29:44.550648 13515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:29:44.553233 13532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:44.554524 13515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 22:29:44.554664 13515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "e3fc32d9ad86432e873fb64ecf01f237"
format_stamp: "Formatted at 2026-08-11 22:29:44 on dist-test-slave-mvvj"
server_key: "cb8bd2aaf881d2cd6bc46152fdc27602"
server_key_iv: "cf40c262f59eb4c8c086a188ff1c411c"
server_key_version: "encryptionkey@0"
I20260811 22:29:44.554783 13515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:44.571224 13515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:44.572064 13515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:44.572279 13515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:44.572923 13515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:44.573937 13515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:44.574023 13515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:44.574101 13515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:44.574155 13515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:44.584000 13515 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:38969
I20260811 22:29:44.584059 13661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:38969 every 8 connection(s)
I20260811 22:29:44.585057 13515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:29:44.589604 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 13515
I20260811 22:29:44.589766 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:29:44.590070 24863 external_mini_cluster.cc:1428] Setting key e1a1f880d2abf8e741ee4b78d7e85c28
I20260811 22:29:44.596038 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.587201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:53475 (local address 127.24.71.254:32923)
0811 22:29:44.587361 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.587366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.589096 (+ 1730us) server_negotiation.cc:407] Connection header received
0811 22:29:44.590995 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.590999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.591063 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.591148 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:44.592226 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.592767 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.593949 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.594193 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.594306 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.595465 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:44.595477 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:44.595479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:44.595493 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:44.595515 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:44.595523 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:44.595606 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:44.595777 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:29:44.595879 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:29:44.596972 13665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.587724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:32923 (local address 127.24.71.195:53475)
0811 22:29:44.588888 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:44.588939 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:44.590754 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:44.591348 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:44.591355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:44.591448 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:44.592075 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.592092 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.592910 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.592915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.593810 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.593822 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.594374 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.594380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.594545 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.595157 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:29:44.595159 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:44.595204 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:29:44.595278 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:44.595608 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:44.595620 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:44.595629 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:29:44.595777 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 22:29:44.596101 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":139,"threads_started":1}
I20260811 22:29:44.598167 13662 heartbeater.cc:347] Connected to a master server at 127.24.71.254:32923
I20260811 22:29:44.598438 13662 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:44.599120 13662 heartbeater.cc:511] Master 127.24.71.254:32923 requested a full tablet report, sending...
I20260811 22:29:44.600332 13154 ts_manager.cc:194] Registered new tserver with Master: e3fc32d9ad86432e873fb64ecf01f237 (127.24.71.195:38969)
I20260811 22:29:44.600989 13154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.195:53475
I20260811 22:29:44.604565 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:29:44.610723 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.606627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.24.71.254:32923)
0811 22:29:44.606777 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.606783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.606953 (+ 170us) server_negotiation.cc:407] Connection header received
0811 22:29:44.607085 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.607087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.607127 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.607194 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:44.607426 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.608013 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.608879 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.609119 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.609204 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.610092 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:44.610095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:44.610097 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:44.610110 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:44.610132 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:44.610145 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:44.610260 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:44.610478 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 22:29:44.610552 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:29:44.614286 13154 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:29:44.617581 13154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:29:44.633800 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.627414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:46749 (local address 127.0.0.1:45958)
0811 22:29:44.628202 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:44.628256 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:44.628397 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:44.629075 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:44.629080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:44.629103 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:44.629258 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.629266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.630623 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.630627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.632043 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.632051 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.633126 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.633131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.633297 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.633371 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 22:29:44.633460 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:29:44.633587 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":12,"thread_start_us":99,"threads_started":1}
I20260811 22:29:44.633836 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.628623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36380 (local address 127.24.71.194:41235)
0811 22:29:44.629228 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.629242 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.629273 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:29:44.629366 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.629371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.629537 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.629663 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:44.629939 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.630967 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.632379 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.632942 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.633086 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.633321 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:44.633554 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 22:29:44.633674 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":124,"threads_started":1}
I20260811 22:29:44.634096 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.628201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:41235 (local address 127.0.0.1:36380)
0811 22:29:44.628779 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:44.628793 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:44.628881 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:44.629683 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:44.629686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:44.629705 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:44.629787 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.629793 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.631087 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.631093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.632214 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.632226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.633100 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.633104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.633329 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.633383 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 22:29:44.633517 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 22:29:44.633612 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":92,"thread_start_us":89,"threads_started":1}
I20260811 22:29:44.634580 13678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.627646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.24.71.193:46749)
0811 22:29:44.628368 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.628382 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.628422 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:29:44.628637 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.628645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.628885 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.629096 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:44.629403 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.630528 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.632175 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.632970 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.633705 (+ 735us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.633892 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:44.634311 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 22:29:44.634420 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":128,"threads_started":1}
I20260811 22:29:44.634883 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.628507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:38969 (local address 127.0.0.1:39028)
0811 22:29:44.629881 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:44.629898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:44.630020 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:44.630435 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:44.630438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:44.630459 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:44.630549 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.630555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.631978 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.631982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.633138 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.633147 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.634478 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.634482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.634580 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.634603 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:29:44.634681 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:29:44.634752 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1285,"mutex_wait_us":3,"thread_start_us":114,"threads_started":1}
I20260811 22:29:44.635434 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.628118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39028 (local address 127.24.71.195:38969)
0811 22:29:44.628992 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.629007 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.630021 (+ 1014us) server_negotiation.cc:407] Connection header received
0811 22:29:44.630129 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.630134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.630292 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.630420 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:44.630712 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.631848 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.633272 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.634308 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.634384 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.634553 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:44.635145 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 22:29:44.635239 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":186,"threads_started":1}
I20260811 22:29:44.637454 13439 tablet_service.cc:1467] Processing CreateTablet for tablet 9af25fbbf213439e847c8b342dd13e6a (DEFAULT_TABLE table=test-table [id=e90f4def23004448845cd2576e03861e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:44.638547 13439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9af25fbbf213439e847c8b342dd13e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:44.638427 13290 tablet_service.cc:1467] Processing CreateTablet for tablet 9af25fbbf213439e847c8b342dd13e6a (DEFAULT_TABLE table=test-table [id=e90f4def23004448845cd2576e03861e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:44.638427 13584 tablet_service.cc:1467] Processing CreateTablet for tablet 9af25fbbf213439e847c8b342dd13e6a (DEFAULT_TABLE table=test-table [id=e90f4def23004448845cd2576e03861e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:44.639575 13584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9af25fbbf213439e847c8b342dd13e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:44.639575 13290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9af25fbbf213439e847c8b342dd13e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:44.645823 13684 tablet_bootstrap.cc:492] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: Bootstrap starting.
I20260811 22:29:44.647753 13686 tablet_bootstrap.cc:492] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: Bootstrap starting.
I20260811 22:29:44.648515 13685 tablet_bootstrap.cc:492] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2: Bootstrap starting.
I20260811 22:29:44.648872 13684 tablet_bootstrap.cc:654] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:44.650519 13685 tablet_bootstrap.cc:654] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:44.650648 13686 tablet_bootstrap.cc:654] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:44.651155 13684 log.cc:824] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:44.651345 13685 log.cc:824] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:44.651499 13686 log.cc:824] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:44.653688 13685 tablet_bootstrap.cc:492] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2: No bootstrap required, opened a new log
I20260811 22:29:44.653954 13685 ts_tablet_manager.cc:1397] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 22:29:44.654168 13686 tablet_bootstrap.cc:492] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: No bootstrap required, opened a new log
I20260811 22:29:44.654230 13684 tablet_bootstrap.cc:492] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: No bootstrap required, opened a new log
I20260811 22:29:44.654388 13686 ts_tablet_manager.cc:1397] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260811 22:29:44.654500 13684 ts_tablet_manager.cc:1397] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 22:29:44.657980 13686 raft_consensus.cc:348] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } }
I20260811 22:29:44.658257 13686 raft_consensus.cc:374] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:44.658344 13686 raft_consensus.cc:729] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3fc32d9ad86432e873fb64ecf01f237, State: Initialized, Role: FOLLOWER
I20260811 22:29:44.658475 13684 raft_consensus.cc:348] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } }
I20260811 22:29:44.658757 13684 raft_consensus.cc:374] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:44.658814 13684 raft_consensus.cc:729] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9e0b8d45f1419f86ac25002248e9f9, State: Initialized, Role: FOLLOWER
I20260811 22:29:44.658888 13686 consensus_queue.cc:260] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } }
I20260811 22:29:44.659264 13684 consensus_queue.cc:260] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } }
I20260811 22:29:44.659852 13685 raft_consensus.cc:348] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } }
I20260811 22:29:44.660064 13686 ts_tablet_manager.cc:1428] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 22:29:44.660102 13512 heartbeater.cc:503] Master 127.24.71.254:32923 was elected leader, sending a full tablet report...
I20260811 22:29:44.660107 13685 raft_consensus.cc:374] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:44.660161 13685 raft_consensus.cc:729] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2590b1fd03304e21afe4b4a403b46fb2, State: Initialized, Role: FOLLOWER
I20260811 22:29:44.660415 13662 heartbeater.cc:503] Master 127.24.71.254:32923 was elected leader, sending a full tablet report...
I20260811 22:29:44.660624 13685 consensus_queue.cc:260] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } }
I20260811 22:29:44.661427 13368 heartbeater.cc:503] Master 127.24.71.254:32923 was elected leader, sending a full tablet report...
I20260811 22:29:44.661610 13684 ts_tablet_manager.cc:1428] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:29:44.663388 13685 ts_tablet_manager.cc:1428] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
W20260811 22:29:44.684762 13369 tablet.cc:2369] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:29:44.836381 13664 tablet.cc:2369] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:29:44.878521 13513 tablet.cc:2369] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:29:44.910104 13692 raft_consensus.cc:482] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:44.910358 13692 raft_consensus.cc:504] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } }
I20260811 22:29:44.911623 13692 leader_election.cc:288] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2590b1fd03304e21afe4b4a403b46fb2 (127.24.71.193:46749), e3fc32d9ad86432e873fb64ecf01f237 (127.24.71.195:38969)
I20260811 22:29:44.915763 13678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.912011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:51747 (local address 127.24.71.193:46749)
0811 22:29:44.912154 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.912159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.912346 (+ 187us) server_negotiation.cc:407] Connection header received
0811 22:29:44.912606 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.912613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.912677 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.912751 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:44.913089 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.913624 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.914766 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.915152 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.915249 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.915336 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:44.915554 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 22:29:44.915633 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:29:44.915795 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.912270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:34617 (local address 127.24.71.195:38969)
0811 22:29:44.912521 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.912525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.912543 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:29:44.912702 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.912704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.912773 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.912849 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:44.913156 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.913650 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.914796 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.915184 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.915320 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.915369 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:44.915620 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 22:29:44.915690 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260811 22:29:44.915776 13694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.911857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:46749 (local address 127.24.71.194:51747)
0811 22:29:44.912249 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:44.912266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:44.912447 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:44.912789 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:44.912794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:44.912818 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:44.912958 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.912970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.913745 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.913750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.914633 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.914642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.915281 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.915285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.915415 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.915445 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:29:44.915514 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:29:44.915602 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":46,"thread_start_us":123,"threads_started":1}
I20260811 22:29:44.915784 13696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.912110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:38969 (local address 127.24.71.194:34617)
0811 22:29:44.912439 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:44.912456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:44.912584 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:44.912895 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:44.912898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:44.912913 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:44.913010 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.913037 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.913795 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.913798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.914637 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:44.914646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.915341 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:44.915345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:44.915463 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:44.915481 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:29:44.915581 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:29:44.915644 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":53,"thread_start_us":91,"threads_started":1}
I20260811 22:29:44.916630 13311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9af25fbbf213439e847c8b342dd13e6a" candidate_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2590b1fd03304e21afe4b4a403b46fb2" is_pre_election: true
I20260811 22:29:44.916697 13606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9af25fbbf213439e847c8b342dd13e6a" candidate_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3fc32d9ad86432e873fb64ecf01f237" is_pre_election: true
I20260811 22:29:44.916952 13311 raft_consensus.cc:2454] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd9e0b8d45f1419f86ac25002248e9f9 in term 0.
I20260811 22:29:44.917016 13606 raft_consensus.cc:2454] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd9e0b8d45f1419f86ac25002248e9f9 in term 0.
I20260811 22:29:44.917519 13389 leader_election.cc:302] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2590b1fd03304e21afe4b4a403b46fb2, dd9e0b8d45f1419f86ac25002248e9f9, e3fc32d9ad86432e873fb64ecf01f237; no voters:
I20260811 22:29:44.917840 13692 raft_consensus.cc:2781] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:29:44.917929 13692 raft_consensus.cc:482] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:29:44.917972 13692 raft_consensus.cc:3037] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:44.919014 13692 raft_consensus.cc:504] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } }
I20260811 22:29:44.919406 13692 leader_election.cc:288] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [CANDIDATE]: Term 1 election: Requested vote from peers 2590b1fd03304e21afe4b4a403b46fb2 (127.24.71.193:46749), e3fc32d9ad86432e873fb64ecf01f237 (127.24.71.195:38969)
I20260811 22:29:44.919749 13311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9af25fbbf213439e847c8b342dd13e6a" candidate_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2590b1fd03304e21afe4b4a403b46fb2"
I20260811 22:29:44.919917 13311 raft_consensus.cc:3037] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:44.919924 13606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9af25fbbf213439e847c8b342dd13e6a" candidate_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3fc32d9ad86432e873fb64ecf01f237"
I20260811 22:29:44.920091 13606 raft_consensus.cc:3037] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:44.921115 13311 raft_consensus.cc:2454] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd9e0b8d45f1419f86ac25002248e9f9 in term 1.
I20260811 22:29:44.921344 13606 raft_consensus.cc:2454] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd9e0b8d45f1419f86ac25002248e9f9 in term 1.
I20260811 22:29:44.921459 13389 leader_election.cc:302] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2590b1fd03304e21afe4b4a403b46fb2, dd9e0b8d45f1419f86ac25002248e9f9; no voters:
I20260811 22:29:44.921659 13692 raft_consensus.cc:2781] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:44.921957 13692 raft_consensus.cc:686] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 1 LEADER]: Becoming Leader. State: Replica: dd9e0b8d45f1419f86ac25002248e9f9, State: Running, Role: LEADER
I20260811 22:29:44.922340 13692 consensus_queue.cc:237] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } }
I20260811 22:29:44.926252 13154 catalog_manager.cc:5495] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd9e0b8d45f1419f86ac25002248e9f9 (127.24.71.194). New cstate: current_term: 1 leader_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd9e0b8d45f1419f86ac25002248e9f9" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 41235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 } health_report { overall_health: UNKNOWN } } }
I20260811 22:29:45.002408 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:44.998511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.24.71.194:41235)
0811 22:29:44.998658 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:44.998662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:44.998676 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:29:44.998747 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:44.998749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:44.998805 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:44.998897 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:29:44.999202 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:44.999767 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:45.000782 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:45.001023 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:45.001116 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:45.001418 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:29:45.001875 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:29:45.001994 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:45.002177 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 22:29:45.002249 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:29:45.009109 13606 raft_consensus.cc:1261] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [term 1 FOLLOWER]: Refusing update from remote peer dd9e0b8d45f1419f86ac25002248e9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:45.009110 13311 raft_consensus.cc:1261] T 9af25fbbf213439e847c8b342dd13e6a P 2590b1fd03304e21afe4b4a403b46fb2 [term 1 FOLLOWER]: Refusing update from remote peer dd9e0b8d45f1419f86ac25002248e9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:45.009876 13692 consensus_queue.cc:1035] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2590b1fd03304e21afe4b4a403b46fb2" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 46749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:45.010113 13697 consensus_queue.cc:1035] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3fc32d9ad86432e873fb64ecf01f237" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 38969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:45.017402 13700 mvcc.cc:205] Tried to move back new op lower bound from 7317452328989392896 to 7317452328644767744. Current Snapshot: MvccSnapshot[applied={T|T < 7317452328989392896}]
I20260811 22:29:45.017603 13702 mvcc.cc:205] Tried to move back new op lower bound from 7317452328989392896 to 7317452328644767744. Current Snapshot: MvccSnapshot[applied={T|T < 7317452328989392896}]
I20260811 22:29:45.019886 13701 mvcc.cc:205] Tried to move back new op lower bound from 7317452328989392896 to 7317452328644767744. Current Snapshot: MvccSnapshot[applied={T|T < 7317452328989392896}]
I20260811 22:29:45.028656 13153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:45.028874 13153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:45.031688 13153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9a00e0a90514fc792eb6a0185d7735f: Sending DeleteTablet for 3 replicas of tablet 9af25fbbf213439e847c8b342dd13e6a
I20260811 22:29:45.032763 13290 tablet_service.cc:1514] Processing DeleteTablet for tablet 9af25fbbf213439e847c8b342dd13e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:45 UTC) from {username='slave'} at 127.0.0.1:45958
I20260811 22:29:45.032873 13439 tablet_service.cc:1514] Processing DeleteTablet for tablet 9af25fbbf213439e847c8b342dd13e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:45 UTC) from {username='slave'} at 127.0.0.1:36380
I20260811 22:29:45.033408 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 13220
I20260811 22:29:45.033396 13584 tablet_service.cc:1514] Processing DeleteTablet for tablet 9af25fbbf213439e847c8b342dd13e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:45 UTC) from {username='slave'} at 127.0.0.1:39028
I20260811 22:29:45.035297 13712 tablet_replica.cc:330] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: stopping tablet replica
I20260811 22:29:45.035707 13712 raft_consensus.cc:2229] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:45.036190 13712 raft_consensus.cc:2258] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:45.038159 13712 ts_tablet_manager.cc:1905] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:45.040927 13712 ts_tablet_manager.cc:1918] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:45.041060 13712 log.cc:1196] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/9af25fbbf213439e847c8b342dd13e6a
I20260811 22:29:45.041298 13711 tablet_replica.cc:330] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: stopping tablet replica
I20260811 22:29:45.041426 13712 ts_tablet_manager.cc:1939] T 9af25fbbf213439e847c8b342dd13e6a P e3fc32d9ad86432e873fb64ecf01f237: Deleting consensus metadata
I20260811 22:29:45.041675 13711 raft_consensus.cc:2229] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:29:45.042124 13711 raft_consensus.cc:2258] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:45.042877 13141 catalog_manager.cc:4841] TS e3fc32d9ad86432e873fb64ecf01f237 (127.24.71.195:38969): tablet 9af25fbbf213439e847c8b342dd13e6a (table test-table [id=e90f4def23004448845cd2576e03861e]) successfully deleted
I20260811 22:29:45.043823 13711 ts_tablet_manager.cc:1905] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:45.046113 13711 ts_tablet_manager.cc:1918] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:45.046234 13711 log.cc:1196] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/9af25fbbf213439e847c8b342dd13e6a
I20260811 22:29:45.046526 13711 ts_tablet_manager.cc:1939] T 9af25fbbf213439e847c8b342dd13e6a P dd9e0b8d45f1419f86ac25002248e9f9: Deleting consensus metadata
W20260811 22:29:45.047384 13137 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:46749 (error 108)
I20260811 22:29:45.047919 13137 catalog_manager.cc:4841] TS dd9e0b8d45f1419f86ac25002248e9f9 (127.24.71.194:41235): tablet 9af25fbbf213439e847c8b342dd13e6a (table test-table [id=e90f4def23004448845cd2576e03861e]) successfully deleted
I20260811 22:29:45.048542 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 13371
W20260811 22:29:45.048574 13679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:29:45.048250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:46749 (local address 127.0.0.1:45968)
0811 22:29:45.048414 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:45.048491 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:46749: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:29:45.052954 13137 catalog_manager.cc:4584] TS 2590b1fd03304e21afe4b4a403b46fb2 (127.24.71.193:46749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9af25fbbf213439e847c8b342dd13e6a: Network error: Client connection negotiation failed: client connection to 127.24.71.193:46749: connect: Connection reset by peer (error 104)
I20260811 22:29:45.058168 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 13515
I20260811 22:29:45.066839 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 13120
2026-08-11T22:29:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1401 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:45 dist-test-slave-mvvj krb5kdc[13716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:45 dist-test-slave-mvvj krb5kdc[13716](info): set up 2 sockets
Aug 11 22:29:45 dist-test-slave-mvvj krb5kdc[13716](info): commencing operation
krb5kdc: starting...
W20260811 22:29:46.178694 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:46 dist-test-slave-mvvj krb5kdc[13716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:29:46.318686 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:45713
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:46249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:46.438637 13731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:46.438948 13731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:46.439085 13731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:46.443128 13731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:46.443222 13731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:46.443264 13731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:46.443301 13731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:46.443336 13731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:46.448565 13731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:46249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:45713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:46.449667 13731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:46.450702 13731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:46.459681 13740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:46.459690 13738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:46.459757 13731 server_base.cc:1034] running on GCE node
W20260811 22:29:46.459689 13737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:46.460680 13731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:46.461889 13731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:46.463057 13731 hybrid_clock.cc:648] HybridClock initialized: now 1786487386463037 us; error 32 us; skew 500 ppm
Aug 11 22:29:46 dist-test-slave-mvvj krb5kdc[13716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487386, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:46.466917 13731 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:29:46.467991 13731 webserver.cc:460] Webserver started at http://127.24.71.254:33555/ using document root <none> and password file <none>
I20260811 22:29:46.468538 13731 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:46.468600 13731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:46.468775 13731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:46.470844 13731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "bd23c7f9b9d243cf881a0cfef27f47a2"
format_stamp: "Formatted at 2026-08-11 22:29:46 on dist-test-slave-mvvj"
server_key: "63b5c97542e1ab6b93b8b294899751c5"
server_key_iv: "419c50cfe4e2039e1032cec17591ac7a"
server_key_version: "encryptionkey@0"
I20260811 22:29:46.471400 13731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "bd23c7f9b9d243cf881a0cfef27f47a2"
format_stamp: "Formatted at 2026-08-11 22:29:46 on dist-test-slave-mvvj"
server_key: "63b5c97542e1ab6b93b8b294899751c5"
server_key_iv: "419c50cfe4e2039e1032cec17591ac7a"
server_key_version: "encryptionkey@0"
I20260811 22:29:46.475191 13731 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 22:29:46.477777 13747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:46.479296 13731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:29:46.479403 13731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "bd23c7f9b9d243cf881a0cfef27f47a2"
format_stamp: "Formatted at 2026-08-11 22:29:46 on dist-test-slave-mvvj"
server_key: "63b5c97542e1ab6b93b8b294899751c5"
server_key_iv: "419c50cfe4e2039e1032cec17591ac7a"
server_key_version: "encryptionkey@0"
I20260811 22:29:46.479550 13731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:46.496565 13731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:46.499998 13731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:46.500205 13731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:46.508147 13731 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:45713
I20260811 22:29:46.508162 13808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:45713 every 8 connection(s)
I20260811 22:29:46.509130 13731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:46.512007 13809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:46.516770 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 13731
I20260811 22:29:46.516925 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:46.517267 24863 external_mini_cluster.cc:1428] Setting key 499fe35f68cb8141b99298bea3bd7bef
I20260811 22:29:46.518246 13809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2: Bootstrap starting.
I20260811 22:29:46.520798 13809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:46.521737 13809 log.cc:824] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:46.524029 13809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2: No bootstrap required, opened a new log
Aug 11 22:29:46 dist-test-slave-mvvj krb5kdc[13716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:46.528628 13809 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd23c7f9b9d243cf881a0cfef27f47a2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45713 } }
I20260811 22:29:46.528888 13809 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:46.528975 13809 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd23c7f9b9d243cf881a0cfef27f47a2, State: Initialized, Role: FOLLOWER
I20260811 22:29:46.529428 13809 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd23c7f9b9d243cf881a0cfef27f47a2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45713 } }
I20260811 22:29:46.529587 13809 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:46.529677 13809 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:46.529801 13809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:46.530767 13809 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd23c7f9b9d243cf881a0cfef27f47a2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45713 } }
I20260811 22:29:46.531126 13809 leader_election.cc:302] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd23c7f9b9d243cf881a0cfef27f47a2; no voters:
I20260811 22:29:46.531514 13809 leader_election.cc:288] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:46.531607 13815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:46.531908 13815 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [term 1 LEADER]: Becoming Leader. State: Replica: bd23c7f9b9d243cf881a0cfef27f47a2, State: Running, Role: LEADER
I20260811 22:29:46.532239 13815 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd23c7f9b9d243cf881a0cfef27f47a2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45713 } }
I20260811 22:29:46.532678 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:46.518742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36616 (local address 127.24.71.254:45713)
0811 22:29:46.519168 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:46.519184 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:46.519236 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:29:46.519964 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:46.519983 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:46.520310 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:46.520586 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:46.521545 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:46.522786 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:46.523960 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:46.524467 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:46.524597 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:46.527361 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:46.527366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:46.527381 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:46.527409 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:46.529318 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:46.529814 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:46.529820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:46.529826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:46.529994 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:46.530324 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:46.530328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:46.530329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:46.530754 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:46.530967 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:46.531293 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 22:29:46.531632 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":140,"threads_started":1}
I20260811 22:29:46.532939 13809 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:46.533505 13815 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd23c7f9b9d243cf881a0cfef27f47a2. Latest consensus state: current_term: 1 leader_uuid: "bd23c7f9b9d243cf881a0cfef27f47a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd23c7f9b9d243cf881a0cfef27f47a2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45713 } } }
I20260811 22:29:46.533619 13815 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:46.534001 13816 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd23c7f9b9d243cf881a0cfef27f47a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd23c7f9b9d243cf881a0cfef27f47a2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45713 } } }
I20260811 22:29:46.534133 13816 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2 [sys.catalog]: This master's current role is: LEADER
W20260811 22:29:46.538067 13831 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:29:46.538137 13831 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:29:46.538195 13820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:46.538898 13820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:46.544165 13820 catalog_manager.cc:1347] Generated new cluster ID: 54218a24c36d4d78b262633ab933f00c
I20260811 22:29:46.544241 13820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:46.572620 13820 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:46.573709 13820 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:46.581178 13820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2: Generated new TSK 0
I20260811 22:29:46.581923 13820 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.71.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
Entry for principal kudu/127.24.71.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab.
I20260811 22:29:46.620338 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45713
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:46249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:46.739663 13837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:46.739969 13837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:46.740018 13837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:46.744283 13837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:46.744355 13837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:46.744454 13837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:29:46.749970 13837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:46249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.193.keytab
--principal=kudu/127.24.71.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:46.751134 13837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:46.752550 13837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:46.762609 13846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:46.762612 13844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:46.762607 13843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:46.762665 13837 server_base.cc:1034] running on GCE node
I20260811 22:29:46.763257 13837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:46.763891 13837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:46.765065 13837 hybrid_clock.cc:648] HybridClock initialized: now 1786487386765032 us; error 44 us; skew 500 ppm
Aug 11 22:29:46 dist-test-slave-mvvj krb5kdc[13716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487386, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:46.768019 13837 init.cc:377] Logged in from keytab as kudu/127.24.71.193@KRBTEST.COM (short username kudu)
I20260811 22:29:46.768929 13837 webserver.cc:460] Webserver started at http://127.24.71.193:34693/ using document root <none> and password file <none>
I20260811 22:29:46.769536 13837 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:46.769632 13837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:46.769870 13837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:46.772042 13837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "f5ea4314199a4383bc2e3bbfa975df76"
format_stamp: "Formatted at 2026-08-11 22:29:46 on dist-test-slave-mvvj"
server_key: "48be79c94a749805ca291e3a6da11560"
server_key_iv: "96bb3709a4d115ff538bc6a92ea78bd9"
server_key_version: "encryptionkey@0"
I20260811 22:29:46.772629 13837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "f5ea4314199a4383bc2e3bbfa975df76"
format_stamp: "Formatted at 2026-08-11 22:29:46 on dist-test-slave-mvvj"
server_key: "48be79c94a749805ca291e3a6da11560"
server_key_iv: "96bb3709a4d115ff538bc6a92ea78bd9"
server_key_version: "encryptionkey@0"
I20260811 22:29:46.776504 13837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:29:46.779232 13854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:46.780615 13837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:46.780757 13837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "f5ea4314199a4383bc2e3bbfa975df76"
format_stamp: "Formatted at 2026-08-11 22:29:46 on dist-test-slave-mvvj"
server_key: "48be79c94a749805ca291e3a6da11560"
server_key_iv: "96bb3709a4d115ff538bc6a92ea78bd9"
server_key_version: "encryptionkey@0"
I20260811 22:29:46.780874 13837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:46.795652 13837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:46.799032 13837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:46.799247 13837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:46.799925 13837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:46.800782 13837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:46.800866 13837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:46.800961 13837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:46.801013 13837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:46.810021 13837 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:39053
I20260811 22:29:46.810050 13985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:39053 every 8 connection(s)
I20260811 22:29:46.810981 13837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:29:46.818486 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 13837
I20260811 22:29:46.818645 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:29:46.818951 24863 external_mini_cluster.cc:1428] Setting key 629453e3605eb22fe0033410478b3f4a
Aug 11 22:29:46 dist-test-slave-mvvj krb5kdc[13716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487386, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.193@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:46.825779 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:46.812968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:53709 (local address 127.24.71.254:45713)
0811 22:29:46.813118 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:46.813122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:46.813944 (+ 822us) server_negotiation.cc:407] Connection header received
0811 22:29:46.815307 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:46.815310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:46.815368 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:46.815450 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:46.816692 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:46.817250 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:46.818423 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:46.818784 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:46.818901 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:46.821748 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:46.821752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:46.821754 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:46.821777 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:46.823474 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:46.824116 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:46.824120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:46.824122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:46.824206 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:46.824578 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:46.824582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:46.824583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:46.824774 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:46.824893 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:46.825463 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 22:29:46.825600 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:29:46.826637 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:46.813350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:45713 (local address 127.24.71.193:53709)
0811 22:29:46.813794 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:46.813837 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:46.815087 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:46.815621 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:46.815628 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:46.816014 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:46.816533 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:46.816547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:46.817408 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:46.817411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:46.818272 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:46.818282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:46.818951 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:46.818956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:46.819172 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:46.819833 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:46.819875 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:46.821569 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:46.823615 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:46.823621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:46.823639 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:46.824011 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:46.824302 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:46.824305 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:46.824307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:46.824471 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:46.824893 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:46.824902 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:46.825175 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 22:29:46.825401 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 22:29:46.825751 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260811 22:29:46.828042 13986 heartbeater.cc:347] Connected to a master server at 127.24.71.254:45713
I20260811 22:29:46.828305 13986 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:46.828841 13986 heartbeater.cc:511] Master 127.24.71.254:45713 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.71.194@KRBTEST.COM; defaulting to no policy
I20260811 22:29:46.830562 13767 ts_manager.cc:194] Registered new tserver with Master: f5ea4314199a4383bc2e3bbfa975df76 (127.24.71.193:39053)
I20260811 22:29:46.831738 13767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.193@KRBTEST.COM'} at 127.24.71.193:53709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
Entry for principal kudu/127.24.71.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab.
I20260811 22:29:46.851053 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45713
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:46249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:46.977690 13992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:46.977983 13992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:46.978034 13992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:46.982218 13992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:46.982295 13992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:46.982402 13992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:29:46.988149 13992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:46249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.194.keytab
--principal=kudu/127.24.71.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:46.989300 13992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:46.990396 13992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:47.000689 13997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:47.000684 13998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:47.000695 14000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:47.001163 13992 server_base.cc:1034] running on GCE node
I20260811 22:29:47.001644 13992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:47.002326 13992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:47.003542 13992 hybrid_clock.cc:648] HybridClock initialized: now 1786487387003501 us; error 52 us; skew 500 ppm
Aug 11 22:29:47 dist-test-slave-mvvj krb5kdc[13716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487387, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:47.006831 13992 init.cc:377] Logged in from keytab as kudu/127.24.71.194@KRBTEST.COM (short username kudu)
I20260811 22:29:47.007798 13992 webserver.cc:460] Webserver started at http://127.24.71.194:35117/ using document root <none> and password file <none>
I20260811 22:29:47.008556 13992 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:47.008659 13992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:47.008905 13992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:47.011099 13992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "eaa26337b6ac42968ba57b77c6503319"
format_stamp: "Formatted at 2026-08-11 22:29:47 on dist-test-slave-mvvj"
server_key: "73d3ae9a9bc8ea6c841b9e69ee299e49"
server_key_iv: "5497a1579a6319dcd75b55f0ba93b677"
server_key_version: "encryptionkey@0"
I20260811 22:29:47.011725 13992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "eaa26337b6ac42968ba57b77c6503319"
format_stamp: "Formatted at 2026-08-11 22:29:47 on dist-test-slave-mvvj"
server_key: "73d3ae9a9bc8ea6c841b9e69ee299e49"
server_key_iv: "5497a1579a6319dcd75b55f0ba93b677"
server_key_version: "encryptionkey@0"
I20260811 22:29:47.015738 13992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:29:47.018711 14008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:47.020310 13992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:47.020471 13992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "eaa26337b6ac42968ba57b77c6503319"
format_stamp: "Formatted at 2026-08-11 22:29:47 on dist-test-slave-mvvj"
server_key: "73d3ae9a9bc8ea6c841b9e69ee299e49"
server_key_iv: "5497a1579a6319dcd75b55f0ba93b677"
server_key_version: "encryptionkey@0"
I20260811 22:29:47.020605 13992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:47.051381 13992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:47.055179 13992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:47.055418 13992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:47.056145 13992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:47.057045 13992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:47.057118 13992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:47.057192 13992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:47.057235 13992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:47.066855 13992 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:43181
I20260811 22:29:47.066907 14134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:43181 every 8 connection(s)
I20260811 22:29:47.067909 13992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:29:47.069756 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 13992
I20260811 22:29:47.069903 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:29:47.070303 24863 external_mini_cluster.cc:1428] Setting key 59f984b0b1e2c046ae31b443c403b463
Aug 11 22:29:47 dist-test-slave-mvvj krb5kdc[13716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487387, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.194@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.71.195@KRBTEST.COM; defaulting to no policy
I20260811 22:29:47.085695 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.070057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:43679 (local address 127.24.71.254:45713)
0811 22:29:47.070242 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:47.070247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:47.071133 (+ 886us) server_negotiation.cc:407] Connection header received
0811 22:29:47.072663 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:47.072669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:47.072734 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:47.072838 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:47.074308 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.074916 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.077006 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.077251 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.077340 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.081051 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:47.081056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:47.081059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:47.081095 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:47.083043 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:47.083685 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:47.083689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:47.083691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:47.083786 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:47.084331 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:47.084335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:47.084341 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:47.084649 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:47.084741 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:47.085403 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 22:29:47.085559 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.195@KRBTEST.COM" created.
I20260811 22:29:47.086802 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.070487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:45713 (local address 127.24.71.194:43679)
0811 22:29:47.070985 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:47.071045 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:47.072434 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:47.073073 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:47.073082 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:47.073555 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:47.074118 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.074134 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.075070 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.075080 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.076838 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.076855 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.077465 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.077472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.077766 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.078625 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:47.078657 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:47.080765 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:47.083178 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:47.083184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:47.083200 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:47.083540 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:47.083937 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:47.083939 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:47.083940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:47.084182 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:47.084753 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:47.084759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:47.085041 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 22:29:47.085246 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 22:29:47.085677 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":143,"threads_started":1}
I20260811 22:29:47.088186 14135 heartbeater.cc:347] Connected to a master server at 127.24.71.254:45713
I20260811 22:29:47.088474 14135 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:47.089044 14135 heartbeater.cc:511] Master 127.24.71.254:45713 requested a full tablet report, sending...
I20260811 22:29:47.090247 13767 ts_manager.cc:194] Registered new tserver with Master: eaa26337b6ac42968ba57b77c6503319 (127.24.71.194:43181)
I20260811 22:29:47.091126 13767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.194@KRBTEST.COM'} at 127.24.71.194:43679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
Entry for principal kudu/127.24.71.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab.
I20260811 22:29:47.106709 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45713
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:46249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:47.271220 14140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:47.271533 14140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:47.271584 14140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:47.276008 14140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:47.276073 14140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:47.276171 14140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:29:47.281970 14140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:46249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.195.keytab
--principal=kudu/127.24.71.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:45713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:47.283190 14140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:47.284394 14140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:47.295078 14146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:47.295073 14145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:47.295367 14148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:47.295475 14140 server_base.cc:1034] running on GCE node
I20260811 22:29:47.296051 14140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:47.296960 14140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:47.298162 14140 hybrid_clock.cc:648] HybridClock initialized: now 1786487387298136 us; error 31 us; skew 500 ppm
Aug 11 22:29:47 dist-test-slave-mvvj krb5kdc[13716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487387, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:47.301970 14140 init.cc:377] Logged in from keytab as kudu/127.24.71.195@KRBTEST.COM (short username kudu)
I20260811 22:29:47.303027 14140 webserver.cc:460] Webserver started at http://127.24.71.195:42915/ using document root <none> and password file <none>
I20260811 22:29:47.303648 14140 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:47.303747 14140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:47.304024 14140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:47.306403 14140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "be769b2434cb420195ed72c5ee0517fb"
format_stamp: "Formatted at 2026-08-11 22:29:47 on dist-test-slave-mvvj"
server_key: "10f6e8ad77bf7ae8e3558634ab4eff32"
server_key_iv: "acb59924ef4de74fcb4a9472a2d406ec"
server_key_version: "encryptionkey@0"
I20260811 22:29:47.307020 14140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "be769b2434cb420195ed72c5ee0517fb"
format_stamp: "Formatted at 2026-08-11 22:29:47 on dist-test-slave-mvvj"
server_key: "10f6e8ad77bf7ae8e3558634ab4eff32"
server_key_iv: "acb59924ef4de74fcb4a9472a2d406ec"
server_key_version: "encryptionkey@0"
I20260811 22:29:47.311163 14140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:29:47.314205 14155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:47.315779 14140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:29:47.315981 14140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "be769b2434cb420195ed72c5ee0517fb"
format_stamp: "Formatted at 2026-08-11 22:29:47 on dist-test-slave-mvvj"
server_key: "10f6e8ad77bf7ae8e3558634ab4eff32"
server_key_iv: "acb59924ef4de74fcb4a9472a2d406ec"
server_key_version: "encryptionkey@0"
I20260811 22:29:47.316120 14140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:47.347821 14140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:47.351428 14140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:47.351655 14140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:47.352669 14140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:47.353591 14140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:47.353667 14140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:47.353744 14140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:47.353789 14140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:47.363606 14140 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:46509
I20260811 22:29:47.363631 14283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:46509 every 8 connection(s)
I20260811 22:29:47.364634 14140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:29:47.366415 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 14140
I20260811 22:29:47.366572 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:29:47.366955 24863 external_mini_cluster.cc:1428] Setting key 3adcc2875d9550c2c97fac1e8164d518
Aug 11 22:29:47 dist-test-slave-mvvj krb5kdc[13716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487387, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.195@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:29:47.380070 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.366847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:49863 (local address 127.24.71.254:45713)
0811 22:29:47.366998 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:47.367002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:47.368256 (+ 1254us) server_negotiation.cc:407] Connection header received
0811 22:29:47.370114 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:47.370117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:47.370175 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:47.370257 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:47.371527 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.372160 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.373365 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.373591 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.373703 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.376380 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:47.376384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:47.376386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:47.376414 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:47.377758 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:47.378283 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:47.378286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:47.378288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:47.378389 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:47.378753 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:47.378756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:47.378757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:47.378957 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:47.379045 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:47.379622 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 22:29:47.379750 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:29:47.381090 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.367360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:45713 (local address 127.24.71.195:49863)
0811 22:29:47.368082 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:47.368148 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:47.369898 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:47.370456 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:47.370463 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:47.370844 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:47.371371 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.371384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.372301 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.372305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.373209 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.373220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.373765 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.373769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.373960 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.374572 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:29:47.374589 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:47.376189 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:47.377890 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:47.377895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:47.377914 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:47.378180 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:47.378482 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:29:47.378485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:29:47.378486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:29:47.378625 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:29:47.379091 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:47.379097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:47.379352 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 22:29:47.379542 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:29:47.379976 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":161,"threads_started":1}
I20260811 22:29:47.382319 14284 heartbeater.cc:347] Connected to a master server at 127.24.71.254:45713
I20260811 22:29:47.382588 14284 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:47.383138 14284 heartbeater.cc:511] Master 127.24.71.254:45713 requested a full tablet report, sending...
I20260811 22:29:47.384305 13767 ts_manager.cc:194] Registered new tserver with Master: be769b2434cb420195ed72c5ee0517fb (127.24.71.195:46509)
I20260811 22:29:47.384894 13767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.71.195@KRBTEST.COM'} at 127.24.71.195:49863
I20260811 22:29:47.393994 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:29:47.404322 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.396201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36620 (local address 127.24.71.254:45713)
0811 22:29:47.396394 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:47.396399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:47.396640 (+ 241us) server_negotiation.cc:407] Connection header received
0811 22:29:47.396806 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:47.396809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:47.396865 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:47.396964 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:47.397711 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.398264 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.399148 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.399400 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.399508 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.401010 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:47.401013 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:47.401016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:47.401051 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:47.402499 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:47.402979 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:47.402983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:47.402985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:47.403073 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:47.403368 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:47.403372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:47.403373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:47.403605 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:47.403755 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:47.404028 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 22:29:47.404150 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:29:47.407716 13767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:29:47.410912 13767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:29:47.426759 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.420849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:43181 (local address 127.0.0.1:35356)
0811 22:29:47.421542 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:47.421557 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:47.421660 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:47.422245 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:47.422248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:47.422264 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:47.422355 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.422361 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.423728 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.423732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.425025 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.425035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.426153 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.426157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.426277 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.426321 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:29:47.426473 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 22:29:47.426604 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":495,"thread_start_us":88,"threads_started":1}
I20260811 22:29:47.426769 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.421048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35356 (local address 127.24.71.194:43181)
0811 22:29:47.421359 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:47.421374 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:47.421671 (+ 297us) server_negotiation.cc:407] Connection header received
0811 22:29:47.421760 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:47.421764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:47.421954 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:47.422102 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:47.422483 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.423590 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.425149 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.426016 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.426140 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.426300 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:47.426507 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 22:29:47.426603 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":131,"threads_started":1}
I20260811 22:29:47.427055 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.420291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:39053 (local address 127.0.0.1:42602)
0811 22:29:47.421960 (+ 1669us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:47.421975 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:47.422074 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:47.422619 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:47.422622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:47.422639 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:47.422723 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.422729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.423744 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.423747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.425229 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.425237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.426534 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.426538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.426675 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.426729 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 22:29:47.426808 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:29:47.426889 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":182,"thread_start_us":172,"threads_started":1}
I20260811 22:29:47.427289 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.420114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:46509 (local address 127.0.0.1:43832)
0811 22:29:47.420604 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:47.420655 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:47.420792 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:47.421872 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:47.421880 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:47.421908 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:47.422059 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.422070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.423588 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.423592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.425195 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.425207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.426534 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.426538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.426721 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.426739 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 22:29:47.426805 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:29:47.427036 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":353,"thread_start_us":78,"threads_started":1}
I20260811 22:29:47.427624 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.420292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.24.71.195:46509)
0811 22:29:47.421136 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:47.421150 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:47.421177 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:29:47.421296 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:47.421305 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:47.421534 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:47.421709 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:47.422206 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.423431 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.425359 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.426222 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.426331 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.426466 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:47.427207 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 22:29:47.427307 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":154,"threads_started":1}
I20260811 22:29:47.427788 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.421036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.24.71.193:39053)
0811 22:29:47.421453 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:47.421460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:47.422169 (+ 709us) server_negotiation.cc:407] Connection header received
0811 22:29:47.422255 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:47.422260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:47.422488 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:47.422639 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:47.422849 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.423606 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.425359 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.426225 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.426331 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.426466 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:47.427534 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0811 22:29:47.427633 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":111,"threads_started":1}
I20260811 22:29:47.430302 13908 tablet_service.cc:1467] Processing CreateTablet for tablet 549af4e24f114261846b6964ee25cffa (DEFAULT_TABLE table=test-table [id=0497017808374657a3dad8b5e6de70ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:47.430233 14060 tablet_service.cc:1467] Processing CreateTablet for tablet 549af4e24f114261846b6964ee25cffa (DEFAULT_TABLE table=test-table [id=0497017808374657a3dad8b5e6de70ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:47.430233 14209 tablet_service.cc:1467] Processing CreateTablet for tablet 549af4e24f114261846b6964ee25cffa (DEFAULT_TABLE table=test-table [id=0497017808374657a3dad8b5e6de70ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:47.431355 14060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 549af4e24f114261846b6964ee25cffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:47.431351 14209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 549af4e24f114261846b6964ee25cffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:47.431358 13908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 549af4e24f114261846b6964ee25cffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:47.438046 14303 tablet_bootstrap.cc:492] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76: Bootstrap starting.
I20260811 22:29:47.439469 14304 tablet_bootstrap.cc:492] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: Bootstrap starting.
I20260811 22:29:47.440191 14305 tablet_bootstrap.cc:492] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: Bootstrap starting.
I20260811 22:29:47.441296 14303 tablet_bootstrap.cc:654] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:47.441766 14304 tablet_bootstrap.cc:654] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:47.442126 14303 log.cc:824] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:47.442904 14305 tablet_bootstrap.cc:654] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:47.443048 14304 log.cc:824] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:47.443688 14305 log.cc:824] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:47.444285 14303 tablet_bootstrap.cc:492] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76: No bootstrap required, opened a new log
I20260811 22:29:47.444505 14303 ts_tablet_manager.cc:1397] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:29:47.445138 14304 tablet_bootstrap.cc:492] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: No bootstrap required, opened a new log
I20260811 22:29:47.445331 14304 ts_tablet_manager.cc:1397] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:29:47.445712 14305 tablet_bootstrap.cc:492] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: No bootstrap required, opened a new log
I20260811 22:29:47.445885 14305 ts_tablet_manager.cc:1397] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:29:47.448711 14303 raft_consensus.cc:348] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } }
I20260811 22:29:47.448993 14303 raft_consensus.cc:374] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:47.449049 14303 raft_consensus.cc:729] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ea4314199a4383bc2e3bbfa975df76, State: Initialized, Role: FOLLOWER
I20260811 22:29:47.449177 14304 raft_consensus.cc:348] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } }
I20260811 22:29:47.449440 14304 raft_consensus.cc:374] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:47.449501 14304 raft_consensus.cc:729] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be769b2434cb420195ed72c5ee0517fb, State: Initialized, Role: FOLLOWER
I20260811 22:29:47.449551 14303 consensus_queue.cc:260] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } }
I20260811 22:29:47.449676 14305 raft_consensus.cc:348] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } }
I20260811 22:29:47.449940 14305 raft_consensus.cc:374] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:47.450009 14305 raft_consensus.cc:729] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaa26337b6ac42968ba57b77c6503319, State: Initialized, Role: FOLLOWER
I20260811 22:29:47.449954 14304 consensus_queue.cc:260] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } }
I20260811 22:29:47.450408 13986 heartbeater.cc:503] Master 127.24.71.254:45713 was elected leader, sending a full tablet report...
I20260811 22:29:47.450414 14305 consensus_queue.cc:260] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } }
I20260811 22:29:47.451072 14135 heartbeater.cc:503] Master 127.24.71.254:45713 was elected leader, sending a full tablet report...
I20260811 22:29:47.451336 14305 ts_tablet_manager.cc:1428] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:29:47.452018 14304 ts_tablet_manager.cc:1428] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260811 22:29:47.452827 14284 heartbeater.cc:503] Master 127.24.71.254:45713 was elected leader, sending a full tablet report...
I20260811 22:29:47.454216 14303 ts_tablet_manager.cc:1428] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
W20260811 22:29:47.562491 13987 tablet.cc:2369] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:29:47.569308 14136 tablet.cc:2369] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:29:47.615721 14286 tablet.cc:2369] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:29:47.742689 14311 raft_consensus.cc:482] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:47.742980 14311 raft_consensus.cc:504] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } }
I20260811 22:29:47.744220 14311 leader_election.cc:288] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be769b2434cb420195ed72c5ee0517fb (127.24.71.195:46509), f5ea4314199a4383bc2e3bbfa975df76 (127.24.71.193:39053)
I20260811 22:29:47.748324 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.744689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:41021 (local address 127.24.71.193:39053)
0811 22:29:47.744843 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:47.744848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:47.745136 (+ 288us) server_negotiation.cc:407] Connection header received
0811 22:29:47.745295 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:47.745298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:47.745349 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:47.745448 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:47.745752 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.746239 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.747320 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.747730 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.747886 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.747944 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:47.748126 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 22:29:47.748199 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:29:47.748346 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.744538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:39053 (local address 127.24.71.194:41021)
0811 22:29:47.745023 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:47.745035 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:47.745173 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:47.745459 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:47.745461 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:47.745503 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:47.745638 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.745644 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.746356 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.746361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.747189 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.747198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.747887 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.747891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.747989 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.748013 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:29:47.748110 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 22:29:47.748184 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":87,"spinlock_wait_cycles":2688,"thread_start_us":105,"threads_started":1}
I20260811 22:29:47.748452 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.744751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:47135 (local address 127.24.71.195:46509)
0811 22:29:47.744919 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:47.744925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:47.745034 (+ 109us) server_negotiation.cc:407] Connection header received
0811 22:29:47.745215 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:47.745223 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:47.745268 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:47.745358 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:47.745713 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.746237 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.747356 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.747779 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.747979 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.748037 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:47.748240 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:29:47.748310 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:29:47.748502 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.744538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:46509 (local address 127.24.71.194:47135)
0811 22:29:47.744917 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:47.744933 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:47.745095 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:47.745432 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:47.745436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:47.745453 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:47.745590 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.745595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.746368 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.746372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.747224 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:47.747234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.747999 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.748003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:47.748113 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.748130 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:29:47.748222 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:29:47.748295 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":46,"spinlock_wait_cycles":18688,"thread_start_us":79,"threads_started":1}
I20260811 22:29:47.749441 14232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "549af4e24f114261846b6964ee25cffa" candidate_uuid: "eaa26337b6ac42968ba57b77c6503319" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be769b2434cb420195ed72c5ee0517fb" is_pre_election: true
I20260811 22:29:47.749439 13933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "549af4e24f114261846b6964ee25cffa" candidate_uuid: "eaa26337b6ac42968ba57b77c6503319" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ea4314199a4383bc2e3bbfa975df76" is_pre_election: true
I20260811 22:29:47.749768 13933 raft_consensus.cc:2454] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaa26337b6ac42968ba57b77c6503319 in term 0.
I20260811 22:29:47.749773 14232 raft_consensus.cc:2454] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaa26337b6ac42968ba57b77c6503319 in term 0.
I20260811 22:29:47.750293 14011 leader_election.cc:302] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be769b2434cb420195ed72c5ee0517fb, eaa26337b6ac42968ba57b77c6503319; no voters:
I20260811 22:29:47.750587 14311 raft_consensus.cc:2781] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:29:47.750667 14311 raft_consensus.cc:482] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:29:47.750697 14311 raft_consensus.cc:3037] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:47.751888 14311 raft_consensus.cc:504] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } }
I20260811 22:29:47.752291 14311 leader_election.cc:288] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [CANDIDATE]: Term 1 election: Requested vote from peers be769b2434cb420195ed72c5ee0517fb (127.24.71.195:46509), f5ea4314199a4383bc2e3bbfa975df76 (127.24.71.193:39053)
I20260811 22:29:47.752683 14232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "549af4e24f114261846b6964ee25cffa" candidate_uuid: "eaa26337b6ac42968ba57b77c6503319" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be769b2434cb420195ed72c5ee0517fb"
I20260811 22:29:47.752856 14232 raft_consensus.cc:3037] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:47.754077 14232 raft_consensus.cc:2454] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaa26337b6ac42968ba57b77c6503319 in term 1.
I20260811 22:29:47.754426 13933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "549af4e24f114261846b6964ee25cffa" candidate_uuid: "eaa26337b6ac42968ba57b77c6503319" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ea4314199a4383bc2e3bbfa975df76"
I20260811 22:29:47.754495 14011 leader_election.cc:302] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be769b2434cb420195ed72c5ee0517fb, eaa26337b6ac42968ba57b77c6503319; no voters:
I20260811 22:29:47.754590 13933 raft_consensus.cc:3037] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:47.754714 14311 raft_consensus.cc:2781] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:47.754983 14311 raft_consensus.cc:686] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 1 LEADER]: Becoming Leader. State: Replica: eaa26337b6ac42968ba57b77c6503319, State: Running, Role: LEADER
I20260811 22:29:47.755369 14311 consensus_queue.cc:237] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } }
I20260811 22:29:47.756086 13933 raft_consensus.cc:2454] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaa26337b6ac42968ba57b77c6503319 in term 1.
I20260811 22:29:47.758944 13765 catalog_manager.cc:5495] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaa26337b6ac42968ba57b77c6503319 (127.24.71.194). New cstate: current_term: 1 leader_uuid: "eaa26337b6ac42968ba57b77c6503319" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaa26337b6ac42968ba57b77c6503319" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260811 22:29:47.796597 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.792402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.24.71.194:43181)
0811 22:29:47.792536 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:47.792540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:47.792553 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:29:47.792630 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:47.792633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:47.792690 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:47.792788 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:29:47.793091 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.793672 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.794855 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:47.795107 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:47.795203 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:47.795646 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:29:47.796171 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:29:47.796295 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:47.796418 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 22:29:47.796478 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:29:47.802987 14232 raft_consensus.cc:1261] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [term 1 FOLLOWER]: Refusing update from remote peer eaa26337b6ac42968ba57b77c6503319: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:47.802987 13933 raft_consensus.cc:1261] T 549af4e24f114261846b6964ee25cffa P f5ea4314199a4383bc2e3bbfa975df76 [term 1 FOLLOWER]: Refusing update from remote peer eaa26337b6ac42968ba57b77c6503319: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:47.803922 14315 consensus_queue.cc:1035] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ea4314199a4383bc2e3bbfa975df76" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:47.804160 14311 consensus_queue.cc:1035] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be769b2434cb420195ed72c5ee0517fb" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 46509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:47.814468 14322 mvcc.cc:205] Tried to move back new op lower bound from 7317452340433244160 to 7317452340249276416. Current Snapshot: MvccSnapshot[applied={T|T < 7317452340433244160 or (T in {7317452340433244160})}]
I20260811 22:29:47.819965 14321 mvcc.cc:205] Tried to move back new op lower bound from 7317452340433244160 to 7317452340249276416. Current Snapshot: MvccSnapshot[applied={T|T < 7317452340433244160}]
I20260811 22:29:47.827981 13765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:47.828177 13765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:47.830621 13765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd23c7f9b9d243cf881a0cfef27f47a2: Sending DeleteTablet for 3 replicas of tablet 549af4e24f114261846b6964ee25cffa
I20260811 22:29:47.831489 14209 tablet_service.cc:1514] Processing DeleteTablet for tablet 549af4e24f114261846b6964ee25cffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:47 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:43832
I20260811 22:29:47.831518 14060 tablet_service.cc:1514] Processing DeleteTablet for tablet 549af4e24f114261846b6964ee25cffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:47 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:35356
I20260811 22:29:47.831625 13908 tablet_service.cc:1514] Processing DeleteTablet for tablet 549af4e24f114261846b6964ee25cffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:47 UTC) from {username='kudu', principal='kudu/127.24.71.254@KRBTEST.COM'} at 127.0.0.1:42602
I20260811 22:29:47.832106 14333 tablet_replica.cc:330] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: stopping tablet replica
I20260811 22:29:47.832353 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 13837
I20260811 22:29:47.840206 14333 raft_consensus.cc:2229] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:47.840723 14333 raft_consensus.cc:2258] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:47.841207 14332 tablet_replica.cc:330] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: stopping tablet replica
I20260811 22:29:47.841570 14332 raft_consensus.cc:2229] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:29:47.842031 14332 raft_consensus.cc:2258] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:47.842846 14333 ts_tablet_manager.cc:1905] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:47.843498 14332 ts_tablet_manager.cc:1905] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:47.845912 14333 ts_tablet_manager.cc:1918] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:47.846091 14333 log.cc:1196] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/549af4e24f114261846b6964ee25cffa
I20260811 22:29:47.846503 14333 ts_tablet_manager.cc:1939] T 549af4e24f114261846b6964ee25cffa P be769b2434cb420195ed72c5ee0517fb: Deleting consensus metadata
I20260811 22:29:47.847275 14332 ts_tablet_manager.cc:1918] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:47.847391 14332 log.cc:1196] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/549af4e24f114261846b6964ee25cffa
I20260811 22:29:47.847790 14332 ts_tablet_manager.cc:1939] T 549af4e24f114261846b6964ee25cffa P eaa26337b6ac42968ba57b77c6503319: Deleting consensus metadata
W20260811 22:29:47.848183 13748 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:39053 (error 108)
I20260811 22:29:47.848357 13750 catalog_manager.cc:4841] TS be769b2434cb420195ed72c5ee0517fb (127.24.71.195:46509): tablet 549af4e24f114261846b6964ee25cffa (table test-table [id=0497017808374657a3dad8b5e6de70ad]) successfully deleted
I20260811 22:29:47.849088 13748 catalog_manager.cc:4841] TS eaa26337b6ac42968ba57b77c6503319 (127.24.71.194:43181): tablet 549af4e24f114261846b6964ee25cffa (table test-table [id=0497017808374657a3dad8b5e6de70ad]) successfully deleted
I20260811 22:29:47.849162 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 13992
I20260811 22:29:47.851634 14297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:47.850776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:39053 (local address 127.0.0.1:42612)
0811 22:29:47.851484 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:47.851554 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:39053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622}
W20260811 22:29:47.854043 13748 catalog_manager.cc:4584] TS f5ea4314199a4383bc2e3bbfa975df76 (127.24.71.193:39053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 549af4e24f114261846b6964ee25cffa: Network error: Client connection negotiation failed: client connection to 127.24.71.193:39053: connect: Connection refused (error 111)
I20260811 22:29:47.861507 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 14140
I20260811 22:29:47.872084 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 13731
2026-08-11T22:29:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2804 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T22:29:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:47Z Disabled control of system clock
I20260811 22:29:47.906850 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:46523
--webserver_interface=127.24.71.254
--webserver_port=0
--builtin_ntp_servers=127.24.71.212:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:46523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:29:48.023053 14341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:48.023337 14341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:48.023375 14341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:48.027454 14341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:48.027519 14341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:48.027535 14341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:48.027552 14341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:48.027568 14341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:48.032366 14341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:46523
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:46523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:48.033581 14341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:48.034996 14341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:48.043918 14351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:48.043963 14348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:48.043918 14349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:48.043989 14341 server_base.cc:1034] running on GCE node
I20260811 22:29:48.044909 14341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:48.046113 14341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:48.047331 14341 hybrid_clock.cc:648] HybridClock initialized: now 1786487388047305 us; error 50 us; skew 500 ppm
I20260811 22:29:48.054433 14341 webserver.cc:460] Webserver started at http://127.24.71.254:45967/ using document root <none> and password file <none>
I20260811 22:29:48.055119 14341 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:48.055229 14341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:48.055478 14341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:48.057780 14341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "ecc3aeef3da64bd1b4c356333a588db2"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "43885ee46f1b053749ea5334f53fb76e"
server_key_iv: "93ae173dcbec5eff9288b34cc0d2db0b"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.058413 14341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "ecc3aeef3da64bd1b4c356333a588db2"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "43885ee46f1b053749ea5334f53fb76e"
server_key_iv: "93ae173dcbec5eff9288b34cc0d2db0b"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.062461 14341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:29:48.065274 14358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.066641 14341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:29:48.066817 14341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "ecc3aeef3da64bd1b4c356333a588db2"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "43885ee46f1b053749ea5334f53fb76e"
server_key_iv: "93ae173dcbec5eff9288b34cc0d2db0b"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.066955 14341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:48.090948 14341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:48.091935 14341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:48.092165 14341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:48.100405 14341 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:46523
I20260811 22:29:48.100437 14419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:46523 every 8 connection(s)
I20260811 22:29:48.101508 14341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:48.103978 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 14341
I20260811 22:29:48.104140 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:48.104521 24863 external_mini_cluster.cc:1428] Setting key 69a274ce45312f1d63c0791edf159d44
I20260811 22:29:48.105281 14420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:48.112725 14420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2: Bootstrap starting.
I20260811 22:29:48.114655 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.106319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37186 (local address 127.24.71.254:46523)
0811 22:29:48.107008 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:48.107025 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:48.107061 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:29:48.107903 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:48.107925 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:48.108386 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:48.108742 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:48.109185 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.110152 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.111072 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.111554 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.111731 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.112739 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:48.112744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:48.112757 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:48.113231 (+ 474us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:48.113290 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:48.113314 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:48.113434 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:48.113589 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 22:29:48.113894 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":151,"threads_started":1}
I20260811 22:29:48.115325 14420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:48.116298 14420 log.cc:824] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:48.118677 14420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2: No bootstrap required, opened a new log
I20260811 22:29:48.122785 14420 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc3aeef3da64bd1b4c356333a588db2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46523 } }
I20260811 22:29:48.123049 14420 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:48.123087 14420 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecc3aeef3da64bd1b4c356333a588db2, State: Initialized, Role: FOLLOWER
I20260811 22:29:48.123489 14420 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc3aeef3da64bd1b4c356333a588db2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46523 } }
I20260811 22:29:48.123608 14420 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:48.123659 14420 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:48.123733 14420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:48.125116 14420 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc3aeef3da64bd1b4c356333a588db2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46523 } }
I20260811 22:29:48.125455 14420 leader_election.cc:302] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecc3aeef3da64bd1b4c356333a588db2; no voters:
I20260811 22:29:48.125748 14420 leader_election.cc:288] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:48.125988 14427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:48.126325 14427 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [term 1 LEADER]: Becoming Leader. State: Replica: ecc3aeef3da64bd1b4c356333a588db2, State: Running, Role: LEADER
I20260811 22:29:48.126698 14427 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc3aeef3da64bd1b4c356333a588db2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46523 } }
I20260811 22:29:48.127272 14420 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:48.129027 14428 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecc3aeef3da64bd1b4c356333a588db2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc3aeef3da64bd1b4c356333a588db2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46523 } } }
I20260811 22:29:48.129217 14428 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:48.129518 14429 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecc3aeef3da64bd1b4c356333a588db2. Latest consensus state: current_term: 1 leader_uuid: "ecc3aeef3da64bd1b4c356333a588db2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc3aeef3da64bd1b4c356333a588db2" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46523 } } }
I20260811 22:29:48.129622 14429 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:48.129776 14436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:48.133212 14436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:48.140000 14436 catalog_manager.cc:1347] Generated new cluster ID: 49897737628d45b088d672f4b1543359
I20260811 22:29:48.140112 14436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:48.152951 14436 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:48.153931 14436 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:48.161234 14436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2: Generated new TSK 0
I20260811 22:29:48.162088 14436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:29:48.169037 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.193:0
--local_ip_for_outbound_sockets=127.24.71.193
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46523
--builtin_ntp_servers=127.24.71.212:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:29:48.302783 14449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:48.303094 14449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:48.303138 14449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:48.307468 14449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:48.307539 14449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:48.307598 14449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.193
I20260811 22:29:48.312739 14449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.71.193
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.193
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:48.313939 14449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:48.315366 14449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:48.326032 14456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:48.326071 14459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:48.326040 14455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:48.326357 14449 server_base.cc:1034] running on GCE node
I20260811 22:29:48.326898 14449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:48.327675 14449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:48.328847 14449 hybrid_clock.cc:648] HybridClock initialized: now 1786487388328817 us; error 41 us; skew 500 ppm
I20260811 22:29:48.330968 14449 webserver.cc:460] Webserver started at http://127.24.71.193:38011/ using document root <none> and password file <none>
I20260811 22:29:48.331666 14449 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:48.331765 14449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:48.332101 14449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:48.334373 14449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/data/instance:
uuid: "e3748efbc4a4468490e29459d916635f"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "70767410a8c04a21a564690b6c92a90e"
server_key_iv: "9f9326009ed1d1a6781431d16a5ec974"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.335035 14449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance:
uuid: "e3748efbc4a4468490e29459d916635f"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "70767410a8c04a21a564690b6c92a90e"
server_key_iv: "9f9326009ed1d1a6781431d16a5ec974"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.339382 14449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:29:48.342450 14467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.344026 14449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:29:48.344197 14449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/wal
uuid: "e3748efbc4a4468490e29459d916635f"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "70767410a8c04a21a564690b6c92a90e"
server_key_iv: "9f9326009ed1d1a6781431d16a5ec974"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.344342 14449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:48.363452 14449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:48.364650 14449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:48.364925 14449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:48.365723 14449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:48.366899 14449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:48.366979 14449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.367058 14449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:48.367105 14449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.376752 14449 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.193:40527
I20260811 22:29:48.376780 14608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.193:40527 every 8 connection(s)
I20260811 22:29:48.377806 14449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/data/info.pb
I20260811 22:29:48.378631 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 14449
I20260811 22:29:48.378788 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-0/wal/instance
I20260811 22:29:48.379109 24863 external_mini_cluster.cc:1428] Setting key 5a5c5e3a82ea600b8f4e432146b88324
I20260811 22:29:48.381330 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.194:0
--local_ip_for_outbound_sockets=127.24.71.194
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46523
--builtin_ntp_servers=127.24.71.212:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:29:48.388047 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.380018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:60873 (local address 127.24.71.254:46523)
0811 22:29:48.380151 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:48.380156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:48.381495 (+ 1339us) server_negotiation.cc:407] Connection header received
0811 22:29:48.382981 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:48.382984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:48.383049 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:48.383131 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:48.384135 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.384683 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.385861 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.386105 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.386221 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.387492 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:48.387496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:48.387499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:48.387513 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:48.387541 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:48.387550 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:48.387629 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:48.387800 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:29:48.387895 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:29:48.388916 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.380517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:46523 (local address 127.24.71.193:60873)
0811 22:29:48.381327 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:48.381382 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:48.382761 (+ 1379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:48.383322 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:48.383329 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:48.383416 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:48.383985 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.384000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.384826 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.384830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.385721 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.385732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.386271 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.386275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.386452 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.387140 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:29:48.387141 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:48.387183 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:29:48.387249 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:48.387645 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:48.387657 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:48.387668 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 22:29:48.387796 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 22:29:48.388112 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":170,"threads_started":1}
I20260811 22:29:48.390422 14609 heartbeater.cc:347] Connected to a master server at 127.24.71.254:46523
I20260811 22:29:48.390686 14609 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:48.391266 14609 heartbeater.cc:511] Master 127.24.71.254:46523 requested a full tablet report, sending...
I20260811 22:29:48.392921 14378 ts_manager.cc:194] Registered new tserver with Master: e3748efbc4a4468490e29459d916635f (127.24.71.193:40527)
I20260811 22:29:48.394069 14378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.193:60873
W20260811 22:29:48.502557 14612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:48.502905 14612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:48.503012 14612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:48.507181 14612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:48.507272 14612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:48.507359 14612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.194
I20260811 22:29:48.512394 14612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.71.194
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.194
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:48.513581 14612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:48.515036 14612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:48.525547 14619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:48.525563 14622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:48.525549 14618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:48.525887 14612 server_base.cc:1034] running on GCE node
I20260811 22:29:48.526346 14612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:48.527120 14612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:48.528393 14612 hybrid_clock.cc:648] HybridClock initialized: now 1786487388528292 us; error 62 us; skew 500 ppm
I20260811 22:29:48.530507 14612 webserver.cc:460] Webserver started at http://127.24.71.194:41033/ using document root <none> and password file <none>
I20260811 22:29:48.531177 14612 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:48.531276 14612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:48.531533 14612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:48.533743 14612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/data/instance:
uuid: "3033c9989e374b99b71e8ec30872a8d1"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "961952c9a0116d668f573c3eb7334cbd"
server_key_iv: "7381d7afaf1ecf694b0260646ce2a819"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.534349 14612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance:
uuid: "3033c9989e374b99b71e8ec30872a8d1"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "961952c9a0116d668f573c3eb7334cbd"
server_key_iv: "7381d7afaf1ecf694b0260646ce2a819"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.538508 14612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:29:48.541375 14628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.542827 14612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:29:48.542989 14612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/wal
uuid: "3033c9989e374b99b71e8ec30872a8d1"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "961952c9a0116d668f573c3eb7334cbd"
server_key_iv: "7381d7afaf1ecf694b0260646ce2a819"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.543121 14612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:48.552994 14612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:48.553686 14612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:48.553906 14612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:48.554529 14612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:48.555513 14612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:48.555586 14612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.555660 14612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:48.555706 14612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.565709 14612 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.194:38683
I20260811 22:29:48.565740 14763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.194:38683 every 8 connection(s)
I20260811 22:29:48.566679 14612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/data/info.pb
I20260811 22:29:48.569057 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 14612
I20260811 22:29:48.569195 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/wal/instance
I20260811 22:29:48.569540 24863 external_mini_cluster.cc:1428] Setting key bc3378e38a3b474ca57d16149d196697
I20260811 22:29:48.571626 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.71.195:0
--local_ip_for_outbound_sockets=127.24.71.195
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46523
--builtin_ntp_servers=127.24.71.212:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:29:48.577446 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.568819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.194:42869 (local address 127.24.71.254:46523)
0811 22:29:48.569019 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:48.569024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:48.570261 (+ 1237us) server_negotiation.cc:407] Connection header received
0811 22:29:48.571761 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:48.571765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:48.571850 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:48.571942 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:48.572930 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.573711 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.575093 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.575309 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.575455 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.576900 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:48.576903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:48.576905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:48.576923 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:48.576944 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:48.576951 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:48.577013 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:48.577226 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 22:29:48.577282 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:29:48.578810 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.569236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:46523 (local address 127.24.71.194:42869)
0811 22:29:48.570101 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:48.570154 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:48.571539 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:48.572167 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:48.572174 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:48.572258 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:48.572771 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.572785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.573851 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.573865 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.574951 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.574967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.575534 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.575541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.575785 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.576553 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:29:48.576556 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:48.576617 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:29:48.576715 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:48.577047 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:48.577060 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:48.577069 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:29:48.577225 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 22:29:48.577583 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":144,"threads_started":1}
I20260811 22:29:48.580185 14764 heartbeater.cc:347] Connected to a master server at 127.24.71.254:46523
I20260811 22:29:48.580480 14764 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:48.581212 14764 heartbeater.cc:511] Master 127.24.71.254:46523 requested a full tablet report, sending...
I20260811 22:29:48.582490 14378 ts_manager.cc:194] Registered new tserver with Master: 3033c9989e374b99b71e8ec30872a8d1 (127.24.71.194:38683)
I20260811 22:29:48.583099 14378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.194:42869
W20260811 22:29:48.692451 14767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:48.692802 14767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:48.692878 14767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:48.697127 14767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:48.697233 14767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:48.697330 14767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.71.195
I20260811 22:29:48.702361 14767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.71.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.71.195
--webserver_port=0
--tserver_master_addrs=127.24.71.254:46523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.71.195
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:48.703760 14767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:48.705314 14767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:48.716241 14773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:48.716226 14772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:48.716221 14776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:48.716428 14767 server_base.cc:1034] running on GCE node
I20260811 22:29:48.717134 14767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:48.717887 14767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:48.719125 14767 hybrid_clock.cc:648] HybridClock initialized: now 1786487388719114 us; error 44 us; skew 500 ppm
I20260811 22:29:48.721241 14767 webserver.cc:460] Webserver started at http://127.24.71.195:45423/ using document root <none> and password file <none>
I20260811 22:29:48.721916 14767 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:48.721987 14767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:48.722218 14767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:48.724519 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/data/instance:
uuid: "a300b2e207d444ccb18e90af03bbf3e8"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "2dbc67e2d52daf30dcc531e988f07877"
server_key_iv: "ba25c28f0e896b6cb30e8368c558aae9"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.725131 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance:
uuid: "a300b2e207d444ccb18e90af03bbf3e8"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "2dbc67e2d52daf30dcc531e988f07877"
server_key_iv: "ba25c28f0e896b6cb30e8368c558aae9"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.729414 14767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:29:48.732396 14782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.733829 14767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:48.733940 14767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/wal
uuid: "a300b2e207d444ccb18e90af03bbf3e8"
format_stamp: "Formatted at 2026-08-11 22:29:48 on dist-test-slave-mvvj"
server_key: "2dbc67e2d52daf30dcc531e988f07877"
server_key_iv: "ba25c28f0e896b6cb30e8368c558aae9"
server_key_version: "encryptionkey@0"
I20260811 22:29:48.734091 14767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:48.755156 14767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:48.756026 14767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:48.756246 14767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:48.756906 14767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:29:48.757944 14767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:29:48.757997 14767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.758068 14767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:29:48.758104 14767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:48.767545 14767 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.195:36243
I20260811 22:29:48.767567 14915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.195:36243 every 8 connection(s)
I20260811 22:29:48.768571 14767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/data/info.pb
I20260811 22:29:48.770787 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 14767
I20260811 22:29:48.770912 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/wal/instance
I20260811 22:29:48.771241 24863 external_mini_cluster.cc:1428] Setting key 07964dc8ff07851af6ef1bc3a2da525d
I20260811 22:29:48.778640 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.770687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.195:37587 (local address 127.24.71.254:46523)
0811 22:29:48.770860 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:48.770867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:48.772174 (+ 1307us) server_negotiation.cc:407] Connection header received
0811 22:29:48.773620 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:48.773623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:48.773682 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:48.773763 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:48.774682 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.775241 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.776491 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.776720 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.776836 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.778101 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:48.778112 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:48.778114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:48.778126 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:48.778146 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:48.778153 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:48.778236 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:48.778428 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 22:29:48.778493 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:29:48.779577 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.771182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:46523 (local address 127.24.71.195:37587)
0811 22:29:48.772027 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:48.772082 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:48.773392 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:48.773937 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:48.773944 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:48.774033 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:29:48.774536 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.774548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.775379 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.775387 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.776352 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.776363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.776882 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.776886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.777055 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.777781 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:29:48.777783 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:29:48.777826 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:29:48.777904 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:29:48.778247 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:29:48.778259 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:29:48.778266 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:29:48.778423 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 22:29:48.778718 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":179,"threads_started":1}
I20260811 22:29:48.780854 14916 heartbeater.cc:347] Connected to a master server at 127.24.71.254:46523
I20260811 22:29:48.781102 14916 heartbeater.cc:464] Registering TS with master...
I20260811 22:29:48.781605 14916 heartbeater.cc:511] Master 127.24.71.254:46523 requested a full tablet report, sending...
I20260811 22:29:48.782725 14378 ts_manager.cc:194] Registered new tserver with Master: a300b2e207d444ccb18e90af03bbf3e8 (127.24.71.195:36243)
I20260811 22:29:48.783283 14378 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.71.195:37587
I20260811 22:29:48.785598 24863 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:29:48.791915 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.787823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.24.71.254:46523)
0811 22:29:48.788026 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:48.788032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:48.788177 (+ 145us) server_negotiation.cc:407] Connection header received
0811 22:29:48.788337 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:48.788340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:48.788397 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:48.788496 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:48.788794 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.789315 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.790184 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.790407 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.790521 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.791395 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:48.791399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:48.791401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:48.791422 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:48.791442 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:48.791450 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:48.791533 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:48.791657 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 22:29:48.791714 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:29:48.795032 14378 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:29:48.798406 14378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:29:48.814193 14930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.808031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40527 (local address 127.0.0.1:43098)
0811 22:29:48.808881 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:48.808941 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:48.809086 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:48.810061 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:48.810068 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:48.810091 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:48.810298 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.810308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.811963 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.811967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.812752 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.812762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.813594 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.813597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.813713 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.813754 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:29:48.813891 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 22:29:48.814018 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":435,"thread_start_us":111,"threads_started":1}
I20260811 22:29:48.814289 14928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.808160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.24.71.193:40527)
0811 22:29:48.809503 (+ 1343us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:48.809521 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:48.809548 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:29:48.809638 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:48.809644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:48.809863 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:48.810038 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:48.810436 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.811795 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.812907 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.813461 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.813592 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.813804 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:48.814005 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 22:29:48.814118 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1215,"thread_start_us":105,"threads_started":1}
I20260811 22:29:48.814929 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.808349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35508 (local address 127.24.71.194:38683)
0811 22:29:48.809560 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:48.809580 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:48.809602 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:29:48.809667 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:48.809671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:48.809893 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:48.810037 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:48.810586 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.811479 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.812825 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.813632 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.813735 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.813904 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:48.814666 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 22:29:48.814774 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1103,"thread_start_us":108,"threads_started":1}
I20260811 22:29:48.814210 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.808031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:38683 (local address 127.0.0.1:35508)
0811 22:29:48.808882 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:48.808940 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:48.809254 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:48.810215 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:48.810218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:48.810234 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:48.810442 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.810451 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.811608 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.811612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.812688 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.812698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.813886 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.813889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.813971 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.813991 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:29:48.814058 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:29:48.814112 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":23,"spinlock_wait_cycles":400256,"thread_start_us":111,"threads_started":1}
I20260811 22:29:48.815922 14931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.808859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:36243 (local address 127.0.0.1:53358)
0811 22:29:48.809543 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:48.809558 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:48.809651 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:48.810377 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:48.810382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:48.810401 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:48.810546 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.810552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.811963 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.811967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.813902 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:48.813914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.815477 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.815484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:48.815584 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.815603 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:29:48.815692 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:29:48.815761 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":51,"thread_start_us":90,"threads_started":1}
I20260811 22:29:48.816011 14932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:48.809308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.24.71.195:36243)
0811 22:29:48.809792 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:48.809802 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:48.809820 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:29:48.809882 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:48.809885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:48.810095 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:48.810240 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:48.810680 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.811797 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.814064 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:48.815097 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:48.815200 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:48.815405 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:48.815727 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 22:29:48.815848 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":154,"threads_started":1}
I20260811 22:29:48.817585 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 3615d0f3dabd4c67ad102bada1447e34 (DEFAULT_TABLE table=test-table [id=4367c58312b8447fa7dcb5c9e06c3915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:48.817879 14689 tablet_service.cc:1467] Processing CreateTablet for tablet 3615d0f3dabd4c67ad102bada1447e34 (DEFAULT_TABLE table=test-table [id=4367c58312b8447fa7dcb5c9e06c3915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:48.818902 14526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3615d0f3dabd4c67ad102bada1447e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:48.818900 14689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3615d0f3dabd4c67ad102bada1447e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:48.818639 14841 tablet_service.cc:1467] Processing CreateTablet for tablet 3615d0f3dabd4c67ad102bada1447e34 (DEFAULT_TABLE table=test-table [id=4367c58312b8447fa7dcb5c9e06c3915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:29:48.819564 14841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3615d0f3dabd4c67ad102bada1447e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:48.825340 14934 tablet_bootstrap.cc:492] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: Bootstrap starting.
I20260811 22:29:48.826978 14935 tablet_bootstrap.cc:492] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f: Bootstrap starting.
I20260811 22:29:48.827440 14936 tablet_bootstrap.cc:492] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: Bootstrap starting.
I20260811 22:29:48.827785 14934 tablet_bootstrap.cc:654] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:48.828728 14934 log.cc:824] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:48.829070 14935 tablet_bootstrap.cc:654] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:48.829599 14936 tablet_bootstrap.cc:654] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:48.829986 14935 log.cc:824] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:48.830411 14936 log.cc:824] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:48.831387 14934 tablet_bootstrap.cc:492] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: No bootstrap required, opened a new log
I20260811 22:29:48.831614 14934 ts_tablet_manager.cc:1397] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 22:29:48.831979 14935 tablet_bootstrap.cc:492] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f: No bootstrap required, opened a new log
I20260811 22:29:48.832185 14935 ts_tablet_manager.cc:1397] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260811 22:29:48.832475 14936 tablet_bootstrap.cc:492] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: No bootstrap required, opened a new log
I20260811 22:29:48.832675 14936 ts_tablet_manager.cc:1397] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:29:48.835088 14934 raft_consensus.cc:348] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } }
I20260811 22:29:48.835335 14934 raft_consensus.cc:374] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:48.835388 14934 raft_consensus.cc:729] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3033c9989e374b99b71e8ec30872a8d1, State: Initialized, Role: FOLLOWER
I20260811 22:29:48.835891 14934 consensus_queue.cc:260] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } }
I20260811 22:29:48.836567 14764 heartbeater.cc:503] Master 127.24.71.254:46523 was elected leader, sending a full tablet report...
I20260811 22:29:48.836539 14935 raft_consensus.cc:348] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } }
I20260811 22:29:48.836786 14935 raft_consensus.cc:374] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:48.836848 14935 raft_consensus.cc:729] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3748efbc4a4468490e29459d916635f, State: Initialized, Role: FOLLOWER
I20260811 22:29:48.836956 14934 ts_tablet_manager.cc:1428] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260811 22:29:48.837311 14935 consensus_queue.cc:260] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } }
I20260811 22:29:48.838007 14609 heartbeater.cc:503] Master 127.24.71.254:46523 was elected leader, sending a full tablet report...
I20260811 22:29:48.838096 14936 raft_consensus.cc:348] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } }
I20260811 22:29:48.838358 14936 raft_consensus.cc:374] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:48.838411 14936 raft_consensus.cc:729] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a300b2e207d444ccb18e90af03bbf3e8, State: Initialized, Role: FOLLOWER
I20260811 22:29:48.838495 14935 ts_tablet_manager.cc:1428] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:29:48.838893 14936 consensus_queue.cc:260] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } }
I20260811 22:29:48.839576 14916 heartbeater.cc:503] Master 127.24.71.254:46523 was elected leader, sending a full tablet report...
I20260811 22:29:48.839819 14936 ts_tablet_manager.cc:1428] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260811 22:29:48.879166 14610 tablet.cc:2369] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:29:49.019729 14917 tablet.cc:2369] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:29:49.068006 14765 tablet.cc:2369] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:29:49.137140 14943 raft_consensus.cc:482] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:29:49.137372 14943 raft_consensus.cc:504] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } }
I20260811 22:29:49.138643 14943 leader_election.cc:288] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3033c9989e374b99b71e8ec30872a8d1 (127.24.71.194:38683), a300b2e207d444ccb18e90af03bbf3e8 (127.24.71.195:36243)
I20260811 22:29:49.144315 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:49.139204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:59847 (local address 127.24.71.194:38683)
0811 22:29:49.139354 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:49.139361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:49.139481 (+ 120us) server_negotiation.cc:407] Connection header received
0811 22:29:49.139664 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:49.139668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:49.139737 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:49.139820 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:49.140251 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.141172 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.142826 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.143501 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.143656 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:49.143714 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:49.144081 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 22:29:49.144162 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:29:49.144321 14932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:49.139585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.71.193:51615 (local address 127.24.71.195:36243)
0811 22:29:49.139717 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:49.139722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:49.139960 (+ 238us) server_negotiation.cc:407] Connection header received
0811 22:29:49.140129 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:49.140132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:49.140198 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:49.140287 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:29:49.140524 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.141304 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.142826 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.143547 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.143663 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:49.143725 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:49.144081 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 22:29:49.144162 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:29:49.144383 14945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:49.139001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.194:38683 (local address 127.24.71.193:59847)
0811 22:29:49.139354 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:49.139371 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:49.139527 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:49.139887 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:49.139894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:49.139920 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:49.140121 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:49.140130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.141289 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.141292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:49.142675 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:49.142688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.143708 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.143712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:49.143907 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:49.143955 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 22:29:49.144049 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:29:49.144180 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":115,"spinlock_wait_cycles":62720,"thread_start_us":153,"threads_started":1}
I20260811 22:29:49.144381 14947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:49.139479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.195:36243 (local address 127.24.71.193:51615)
0811 22:29:49.139840 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:49.139879 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 22:29:49.140005 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:29:49.140309 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:29:49.140311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:29:49.140327 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:29:49.140416 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:49.140420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.141461 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.141465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:49.142675 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:29:49.142688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.143708 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.143712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:29:49.143895 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:49.143957 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 22:29:49.144055 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 22:29:49.144180 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":138,"spinlock_wait_cycles":768,"thread_start_us":97,"threads_started":1}
I20260811 22:29:49.145345 14712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3615d0f3dabd4c67ad102bada1447e34" candidate_uuid: "e3748efbc4a4468490e29459d916635f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3033c9989e374b99b71e8ec30872a8d1" is_pre_election: true
I20260811 22:29:49.145358 14865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3615d0f3dabd4c67ad102bada1447e34" candidate_uuid: "e3748efbc4a4468490e29459d916635f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a300b2e207d444ccb18e90af03bbf3e8" is_pre_election: true
I20260811 22:29:49.145721 14865 raft_consensus.cc:2454] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3748efbc4a4468490e29459d916635f in term 0.
I20260811 22:29:49.145721 14712 raft_consensus.cc:2454] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3748efbc4a4468490e29459d916635f in term 0.
I20260811 22:29:49.146261 14468 leader_election.cc:302] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a300b2e207d444ccb18e90af03bbf3e8, e3748efbc4a4468490e29459d916635f; no voters:
I20260811 22:29:49.146554 14943 raft_consensus.cc:2781] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:29:49.146665 14943 raft_consensus.cc:482] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:29:49.146747 14943 raft_consensus.cc:3037] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:49.147972 14943 raft_consensus.cc:504] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } }
I20260811 22:29:49.148396 14943 leader_election.cc:288] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [CANDIDATE]: Term 1 election: Requested vote from peers 3033c9989e374b99b71e8ec30872a8d1 (127.24.71.194:38683), a300b2e207d444ccb18e90af03bbf3e8 (127.24.71.195:36243)
I20260811 22:29:49.148861 14712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3615d0f3dabd4c67ad102bada1447e34" candidate_uuid: "e3748efbc4a4468490e29459d916635f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3033c9989e374b99b71e8ec30872a8d1"
I20260811 22:29:49.148877 14865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3615d0f3dabd4c67ad102bada1447e34" candidate_uuid: "e3748efbc4a4468490e29459d916635f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a300b2e207d444ccb18e90af03bbf3e8"
I20260811 22:29:49.149021 14865 raft_consensus.cc:3037] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:49.148988 14712 raft_consensus.cc:3037] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:49.150205 14865 raft_consensus.cc:2454] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3748efbc4a4468490e29459d916635f in term 1.
I20260811 22:29:49.150249 14712 raft_consensus.cc:2454] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3748efbc4a4468490e29459d916635f in term 1.
I20260811 22:29:49.150583 14468 leader_election.cc:302] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a300b2e207d444ccb18e90af03bbf3e8, e3748efbc4a4468490e29459d916635f; no voters:
I20260811 22:29:49.150813 14943 raft_consensus.cc:2781] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:49.151091 14943 raft_consensus.cc:686] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [term 1 LEADER]: Becoming Leader. State: Replica: e3748efbc4a4468490e29459d916635f, State: Running, Role: LEADER
I20260811 22:29:49.151464 14943 consensus_queue.cc:237] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } }
I20260811 22:29:49.155117 14378 catalog_manager.cc:5495] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f reported cstate change: term changed from 0 to 1, leader changed from <none> to e3748efbc4a4468490e29459d916635f (127.24.71.193). New cstate: current_term: 1 leader_uuid: "e3748efbc4a4468490e29459d916635f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3748efbc4a4468490e29459d916635f" member_type: VOTER last_known_addr { host: "127.24.71.193" port: 40527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 } health_report { overall_health: UNKNOWN } } }
I20260811 22:29:49.180533 14928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:49.176806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43114 (local address 127.24.71.193:40527)
0811 22:29:49.176962 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:49.176967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:49.176985 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:29:49.177183 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:49.177186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:49.177253 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:49.177358 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:29:49.177597 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.178103 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.178949 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:49.179169 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:49.179253 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:49.179600 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:29:49.180124 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:29:49.180227 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:49.180331 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 22:29:49.180381 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:29:49.187232 14712 raft_consensus.cc:1261] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [term 1 FOLLOWER]: Refusing update from remote peer e3748efbc4a4468490e29459d916635f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:49.187250 14865 raft_consensus.cc:1261] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [term 1 FOLLOWER]: Refusing update from remote peer e3748efbc4a4468490e29459d916635f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:29:49.188146 14948 consensus_queue.cc:1035] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3033c9989e374b99b71e8ec30872a8d1" member_type: VOTER last_known_addr { host: "127.24.71.194" port: 38683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:49.188447 14943 consensus_queue.cc:1035] T 3615d0f3dabd4c67ad102bada1447e34 P e3748efbc4a4468490e29459d916635f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a300b2e207d444ccb18e90af03bbf3e8" member_type: VOTER last_known_addr { host: "127.24.71.195" port: 36243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:29:49.207301 14378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:49.207588 14378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:29:49.210431 14378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecc3aeef3da64bd1b4c356333a588db2: Sending DeleteTablet for 3 replicas of tablet 3615d0f3dabd4c67ad102bada1447e34
I20260811 22:29:49.211349 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 3615d0f3dabd4c67ad102bada1447e34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:49 UTC) from {username='slave'} at 127.0.0.1:43098
I20260811 22:29:49.211453 14841 tablet_service.cc:1514] Processing DeleteTablet for tablet 3615d0f3dabd4c67ad102bada1447e34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:49 UTC) from {username='slave'} at 127.0.0.1:53358
I20260811 22:29:49.211637 14689 tablet_service.cc:1514] Processing DeleteTablet for tablet 3615d0f3dabd4c67ad102bada1447e34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:29:49 UTC) from {username='slave'} at 127.0.0.1:35508
I20260811 22:29:49.212038 14967 tablet_replica.cc:330] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: stopping tablet replica
I20260811 22:29:49.212459 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 14449
I20260811 22:29:49.212471 14967 raft_consensus.cc:2229] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:49.213826 14969 tablet_replica.cc:330] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: stopping tablet replica
I20260811 22:29:49.214205 14969 raft_consensus.cc:2229] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:29:49.214270 14967 raft_consensus.cc:2258] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:49.214601 14969 raft_consensus.cc:2258] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:29:49.217567 14967 ts_tablet_manager.cc:1905] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:49.218369 14969 ts_tablet_manager.cc:1905] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:29:49.220638 14967 ts_tablet_manager.cc:1918] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:49.220744 14967 log.cc:1196] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-2/wal/wals/3615d0f3dabd4c67ad102bada1447e34
I20260811 22:29:49.221078 14967 ts_tablet_manager.cc:1939] T 3615d0f3dabd4c67ad102bada1447e34 P a300b2e207d444ccb18e90af03bbf3e8: Deleting consensus metadata
I20260811 22:29:49.221380 14969 ts_tablet_manager.cc:1918] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:29:49.221486 14969 log.cc:1196] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: Deleting WAL directory at /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786487121090521-24863-0/minicluster-data/ts-1/wal/wals/3615d0f3dabd4c67ad102bada1447e34
I20260811 22:29:49.221799 14969 ts_tablet_manager.cc:1939] T 3615d0f3dabd4c67ad102bada1447e34 P 3033c9989e374b99b71e8ec30872a8d1: Deleting consensus metadata
I20260811 22:29:49.222756 14359 catalog_manager.cc:4841] TS a300b2e207d444ccb18e90af03bbf3e8 (127.24.71.195:36243): tablet 3615d0f3dabd4c67ad102bada1447e34 (table test-table [id=4367c58312b8447fa7dcb5c9e06c3915]) successfully deleted
I20260811 22:29:49.223026 14359 catalog_manager.cc:4841] TS 3033c9989e374b99b71e8ec30872a8d1 (127.24.71.194:38683): tablet 3615d0f3dabd4c67ad102bada1447e34 (table test-table [id=4367c58312b8447fa7dcb5c9e06c3915]) successfully deleted
W20260811 22:29:49.223726 14361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.71.193:40527 (error 108)
I20260811 22:29:49.223933 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 14612
I20260811 22:29:49.228914 14931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:49.228534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.193:40527 (local address 127.0.0.1:43118)
0811 22:29:49.228704 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 22:29:49.228798 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.193:40527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:29:49.229157 14361 catalog_manager.cc:4584] TS e3748efbc4a4468490e29459d916635f (127.24.71.193:40527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3615d0f3dabd4c67ad102bada1447e34: Network error: Client connection negotiation failed: client connection to 127.24.71.193:40527: connect: Connection refused (error 111)
I20260811 22:29:49.233529 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 14767
I20260811 22:29:49.241880 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 14341
2026-08-11T22:29:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1369 ms)
[----------] 6 tests from EncryptionPolicyTest (12119 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:49 dist-test-slave-mvvj krb5kdc[14974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:49 dist-test-slave-mvvj krb5kdc[14974](info): set up 2 sockets
Aug 11 22:29:49 dist-test-slave-mvvj krb5kdc[14974](info): commencing operation
krb5kdc: starting...
W20260811 22:29:50.358510 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:50 dist-test-slave-mvvj krb5kdc[14974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:29:50.496235 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:50.613976 14989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:50.614369 14989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:50.614459 14989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:50.618587 14989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:50.618690 14989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:50.618732 14989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:50.618772 14989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:50.618806 14989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:50.624244 14989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35467
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:50.625418 14989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:50.626524 14989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:50.635251 14999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:50.635263 14996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:50.635262 14997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:50.635543 14989 server_base.cc:1034] running on GCE node
I20260811 22:29:50.636291 14989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:50.637503 14989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:50.638702 14989 hybrid_clock.cc:648] HybridClock initialized: now 1786487390638683 us; error 29 us; skew 500 ppm
Aug 11 22:29:50 dist-test-slave-mvvj krb5kdc[14974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487390, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:50.641646 14989 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:29:50.642735 14989 webserver.cc:460] Webserver started at http://127.0.0.1:39101/ using document root <none> and password file <none>
I20260811 22:29:50.643395 14989 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:50.643505 14989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:50.643767 14989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:50.646102 14989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "981e06f638764cabb8660b710962d7ac"
format_stamp: "Formatted at 2026-08-11 22:29:50 on dist-test-slave-mvvj"
server_key: "beb44066823eb9320d7baec22732c081"
server_key_iv: "42fab1b519baa8a8ad3449da56681502"
server_key_version: "encryptionkey@0"
I20260811 22:29:50.646727 14989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "981e06f638764cabb8660b710962d7ac"
format_stamp: "Formatted at 2026-08-11 22:29:50 on dist-test-slave-mvvj"
server_key: "beb44066823eb9320d7baec22732c081"
server_key_iv: "42fab1b519baa8a8ad3449da56681502"
server_key_version: "encryptionkey@0"
I20260811 22:29:50.650921 14989 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:29:50.653753 15006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:50.655241 14989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:29:50.655373 14989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "981e06f638764cabb8660b710962d7ac"
format_stamp: "Formatted at 2026-08-11 22:29:50 on dist-test-slave-mvvj"
server_key: "beb44066823eb9320d7baec22732c081"
server_key_iv: "42fab1b519baa8a8ad3449da56681502"
server_key_version: "encryptionkey@0"
I20260811 22:29:50.655478 14989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:50.675199 14989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:50.679100 14989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:50.679291 14989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:50.688125 14989 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35467
I20260811 22:29:50.688205 15066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35467 every 8 connection(s)
I20260811 22:29:50.689277 14989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:50.692378 15067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:50.694375 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 14989
I20260811 22:29:50.694542 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:50.694862 24863 external_mini_cluster.cc:1428] Setting key 949e6a4ca8149318275184e80d18eaab
I20260811 22:29:50.699527 15067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac: Bootstrap starting.
I20260811 22:29:50.702140 15067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:50.702936 15067 log.cc:824] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:29:50 dist-test-slave-mvvj krb5kdc[14974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:29:50.705694 15067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac: No bootstrap required, opened a new log
I20260811 22:29:50.709182 15067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981e06f638764cabb8660b710962d7ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35467 } }
I20260811 22:29:50.709522 15067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:50.709618 15067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981e06f638764cabb8660b710962d7ac, State: Initialized, Role: FOLLOWER
I20260811 22:29:50.710076 15067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981e06f638764cabb8660b710962d7ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35467 } }
I20260811 22:29:50.710247 15067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:50.710330 15067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:50.710439 15067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:50.711508 15067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981e06f638764cabb8660b710962d7ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35467 } }
I20260811 22:29:50.711958 15070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:50.696581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44995 (local address 127.0.0.1:35467)
0811 22:29:50.697073 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:50.697089 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:50.697129 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:29:50.697846 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:50.697865 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:50.698222 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:50.698606 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:50.699488 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:50.700628 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:50.701523 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:50.702112 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:50.702295 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:50.705600 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:50.705606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:50.705621 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:50.705651 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:50.708267 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:50.708942 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:50.708947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:50.708953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:50.709071 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:50.709477 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:50.709480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:50.709482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:50.710112 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:50.710371 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:50.710710 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 22:29:50.710981 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":141,"threads_started":1}
I20260811 22:29:50.711927 15067 leader_election.cc:302] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 981e06f638764cabb8660b710962d7ac; no voters:
I20260811 22:29:50.712366 15067 leader_election.cc:288] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:50.712468 15072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:50.712653 15072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [term 1 LEADER]: Becoming Leader. State: Replica: 981e06f638764cabb8660b710962d7ac, State: Running, Role: LEADER
I20260811 22:29:50.712962 15072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981e06f638764cabb8660b710962d7ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35467 } }
I20260811 22:29:50.713781 15067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:50.714658 15073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "981e06f638764cabb8660b710962d7ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981e06f638764cabb8660b710962d7ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35467 } } }
I20260811 22:29:50.714805 15073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:50.715054 15073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 981e06f638764cabb8660b710962d7ac. Latest consensus state: current_term: 1 leader_uuid: "981e06f638764cabb8660b710962d7ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981e06f638764cabb8660b710962d7ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35467 } } }
I20260811 22:29:50.715145 15073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:50.715929 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:29:50.716645 15081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:50.720352 15081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:50.726959 15081 catalog_manager.cc:1347] Generated new cluster ID: d2836ec1d7ea4e6aa93405139fc1c93f
I20260811 22:29:50.727035 15081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:50.733455 15070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:50.718311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55485 (local address 127.0.0.1:35467)
0811 22:29:50.718457 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:50.718462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:50.720136 (+ 1674us) server_negotiation.cc:407] Connection header received
0811 22:29:50.720301 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:50.720304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:50.720358 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:50.720442 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:50.721181 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:50.721721 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:50.722562 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:50.722824 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:50.722933 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:50.724452 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:50.724455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:50.724458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:50.724485 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:50.726334 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:50.732012 (+ 5678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:50.732015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:50.732017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:50.732113 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:50.732509 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:50.732514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:50.732516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:50.732711 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:50.732897 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:50.733168 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 22:29:50.733275 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:29:50.754778 15081 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:50.756179 15081 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:50.761989 15081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 981e06f638764cabb8660b710962d7ac: Generated new TSK 0
I20260811 22:29:50.762771 15081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:29:50.771173 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 14989
W20260811 22:29:50.779975 14971 connection.cc:535] client connection to 127.0.0.1:35467 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35467): Connection reset by peer (error 104)
2026-08-11T22:29:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:50 dist-test-slave-mvvj krb5kdc[15100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:50 dist-test-slave-mvvj krb5kdc[15100](info): set up 2 sockets
Aug 11 22:29:50 dist-test-slave-mvvj krb5kdc[15100](info): commencing operation
krb5kdc: starting...
W20260811 22:29:51.916257 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:52 dist-test-slave-mvvj krb5kdc[15100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:29:52.064119 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45063
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:52.181011 15114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:52.181365 15114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:52.181458 15114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:52.185750 15114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:52.185866 15114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:52.185906 15114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:52.185961 15114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:52.185997 15114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:52.191926 15114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45063
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:52.193157 15114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:52.194242 15114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:52.203709 15120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:52.203706 15123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:52.203917 15114 server_base.cc:1034] running on GCE node
W20260811 22:29:52.203730 15121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:52.204649 15114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:52.205961 15114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:52.207167 15114 hybrid_clock.cc:648] HybridClock initialized: now 1786487392207138 us; error 42 us; skew 500 ppm
Aug 11 22:29:52 dist-test-slave-mvvj krb5kdc[15100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:52.210085 15114 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:29:52.211064 15114 webserver.cc:460] Webserver started at http://127.0.0.1:45605/ using document root <none> and password file <none>
I20260811 22:29:52.211650 15114 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:52.211747 15114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:52.212044 15114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:52.214164 15114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "cfa9558c8e68498393cf7745a7a63fef"
format_stamp: "Formatted at 2026-08-11 22:29:52 on dist-test-slave-mvvj"
server_key: "227b933889fc437b15daa3ad709e5039"
server_key_iv: "d57a1fb6121dcfa2337ea4cb33063fa5"
server_key_version: "encryptionkey@0"
I20260811 22:29:52.214748 15114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "cfa9558c8e68498393cf7745a7a63fef"
format_stamp: "Formatted at 2026-08-11 22:29:52 on dist-test-slave-mvvj"
server_key: "227b933889fc437b15daa3ad709e5039"
server_key_iv: "d57a1fb6121dcfa2337ea4cb33063fa5"
server_key_version: "encryptionkey@0"
I20260811 22:29:52.218865 15114 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:29:52.221539 15132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:52.222873 15114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:29:52.223129 15114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "cfa9558c8e68498393cf7745a7a63fef"
format_stamp: "Formatted at 2026-08-11 22:29:52 on dist-test-slave-mvvj"
server_key: "227b933889fc437b15daa3ad709e5039"
server_key_iv: "d57a1fb6121dcfa2337ea4cb33063fa5"
server_key_version: "encryptionkey@0"
I20260811 22:29:52.223255 15114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:52.249902 15114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:52.253490 15114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:52.253720 15114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:52.261543 15114 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45063
I20260811 22:29:52.261575 15191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45063 every 8 connection(s)
I20260811 22:29:52.262626 15114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:52.265544 15192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:52.271291 15192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef: Bootstrap starting.
I20260811 22:29:52.272387 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 15114
I20260811 22:29:52.272554 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:52.272859 24863 external_mini_cluster.cc:1428] Setting key 0851b912a3d669513ff089875ab47a13
I20260811 22:29:52.274251 15192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:52.275017 15192 log.cc:824] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:52.277254 15192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef: No bootstrap required, opened a new log
I20260811 22:29:52.281818 15192 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa9558c8e68498393cf7745a7a63fef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } }
I20260811 22:29:52.282078 15192 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:52.282173 15192 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfa9558c8e68498393cf7745a7a63fef, State: Initialized, Role: FOLLOWER
I20260811 22:29:52.282250 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:52.274082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53225 (local address 127.0.0.1:45063)
0811 22:29:52.274603 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:52.274619 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:52.274656 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:29:52.275395 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:52.275423 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:52.275910 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:52.276210 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:52.276552 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:52.277561 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:52.278777 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:52.279253 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:52.279378 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:52.280393 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:52.280398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:52.280411 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:52.280808 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:52.280864 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:52.280887 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:52.280990 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:52.281210 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 22:29:52.281372 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":171,"threads_started":1}
I20260811 22:29:52.282697 15192 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa9558c8e68498393cf7745a7a63fef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } }
I20260811 22:29:52.282857 15192 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:52.282945 15192 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:52.283048 15192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:52.284312 15192 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa9558c8e68498393cf7745a7a63fef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } }
I20260811 22:29:52.284695 15192 leader_election.cc:302] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfa9558c8e68498393cf7745a7a63fef; no voters:
I20260811 22:29:52.285094 15192 leader_election.cc:288] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:52.285202 15197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:52.285449 15197 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [term 1 LEADER]: Becoming Leader. State: Replica: cfa9558c8e68498393cf7745a7a63fef, State: Running, Role: LEADER
I20260811 22:29:52.285588 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:29:52.285843 15197 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa9558c8e68498393cf7745a7a63fef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } }
I20260811 22:29:52.286497 15192 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:52.287370 15201 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfa9558c8e68498393cf7745a7a63fef. Latest consensus state: current_term: 1 leader_uuid: "cfa9558c8e68498393cf7745a7a63fef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa9558c8e68498393cf7745a7a63fef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } } }
I20260811 22:29:52.287488 15201 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:52.287854 15199 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfa9558c8e68498393cf7745a7a63fef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa9558c8e68498393cf7745a7a63fef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45063 } } }
I20260811 22:29:52.287948 15199 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:52.289324 15217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:52.292825 15217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:52.293035 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:52.288840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44077 (local address 127.0.0.1:45063)
0811 22:29:52.289042 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:52.289048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:52.289063 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:29:52.289104 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:52.289107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:52.289155 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:52.289250 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:52.289565 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:52.290306 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:52.291154 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:52.291405 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:52.291530 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:52.292419 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:52.292423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:52.292426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:52.292440 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:52.292482 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:52.292494 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:52.292595 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:52.292750 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 22:29:52.292788 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 22:29:52.299302 15217 catalog_manager.cc:1347] Generated new cluster ID: 809d62b1cf024239bcb894f564fccf33
I20260811 22:29:52.299388 15217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:52.320586 15217 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:52.321626 15217 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:52.328080 15217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfa9558c8e68498393cf7745a7a63fef: Generated new TSK 0
I20260811 22:29:52.328878 15217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:29:52.341379 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 15114
W20260811 22:29:52.351064 15098 connection.cc:535] client connection to 127.0.0.1:45063 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45063): Connection reset by peer (error 104)
2026-08-11T22:29:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:52 dist-test-slave-mvvj krb5kdc[15232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:52 dist-test-slave-mvvj krb5kdc[15232](info): set up 2 sockets
Aug 11 22:29:52 dist-test-slave-mvvj krb5kdc[15232](info): commencing operation
krb5kdc: starting...
W20260811 22:29:53.450487 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:53 dist-test-slave-mvvj krb5kdc[15232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:29:53.596455 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:53.715653 15248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:53.715991 15248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:53.716048 15248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:53.720388 15248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:53.720515 15248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:53.720541 15248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:53.720607 15248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:53.720647 15248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:53.726709 15248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33353
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:53.727955 15248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:53.729219 15248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:53.738950 15257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:53.738929 15254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:53.739032 15248 server_base.cc:1034] running on GCE node
W20260811 22:29:53.738929 15255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:53.740011 15248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:53.741482 15248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:53.742735 15248 hybrid_clock.cc:648] HybridClock initialized: now 1786487393742712 us; error 44 us; skew 500 ppm
Aug 11 22:29:53 dist-test-slave-mvvj krb5kdc[15232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:53.749850 15248 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:29:53.751044 15248 webserver.cc:460] Webserver started at http://127.0.0.1:35953/ using document root <none> and password file <none>
I20260811 22:29:53.751771 15248 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:53.751935 15248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:53.752213 15248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:53.754662 15248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "72f303f87dd74969b7d30defe7579a9f"
format_stamp: "Formatted at 2026-08-11 22:29:53 on dist-test-slave-mvvj"
server_key: "47b8551358797aac0f952346a64bb4c5"
server_key_iv: "e865744ec923a2cdd52d594d944960ef"
server_key_version: "encryptionkey@0"
I20260811 22:29:53.755316 15248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "72f303f87dd74969b7d30defe7579a9f"
format_stamp: "Formatted at 2026-08-11 22:29:53 on dist-test-slave-mvvj"
server_key: "47b8551358797aac0f952346a64bb4c5"
server_key_iv: "e865744ec923a2cdd52d594d944960ef"
server_key_version: "encryptionkey@0"
I20260811 22:29:53.759769 15248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 22:29:53.762878 15264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:53.764317 15248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:53.764451 15248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "72f303f87dd74969b7d30defe7579a9f"
format_stamp: "Formatted at 2026-08-11 22:29:53 on dist-test-slave-mvvj"
server_key: "47b8551358797aac0f952346a64bb4c5"
server_key_iv: "e865744ec923a2cdd52d594d944960ef"
server_key_version: "encryptionkey@0"
I20260811 22:29:53.764609 15248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:53.770062 15248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:53.773686 15248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:53.773921 15248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:53.781443 15248 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33353
I20260811 22:29:53.781472 15326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33353 every 8 connection(s)
I20260811 22:29:53.782431 15248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:53.784494 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 15248
I20260811 22:29:53.784632 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:53.784984 24863 external_mini_cluster.cc:1428] Setting key 6d927f397253508625bf096c8c619eef
I20260811 22:29:53.785660 15328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:53.790776 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:53.786484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42165 (local address 127.0.0.1:33353)
0811 22:29:53.786977 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:53.786994 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:53.787034 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:29:53.787695 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:53.787711 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:53.788090 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:53.788435 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:53.789310 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:53.789315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:53.789331 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:53.789342 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:53.789399 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:53.789419 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:53.789575 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:53.789648 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 22:29:53.789809 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":135,"threads_started":1}
I20260811 22:29:53.792809 15328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f: Bootstrap starting.
I20260811 22:29:53.793805 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:29:53.795451 15328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:53.796408 15328 log.cc:824] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:53.798369 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:53.796103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54047 (local address 127.0.0.1:33353)
0811 22:29:53.796221 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:53.796225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:53.797022 (+ 797us) server_negotiation.cc:407] Connection header received
0811 22:29:53.797135 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:53.797141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:53.797184 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:53.797274 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:53.797998 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:53.798001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:53.798002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:53.798013 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:53.798037 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:53.798046 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:53.798116 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:53.798231 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 22:29:53.798264 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:29:53.798940 15328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f: No bootstrap required, opened a new log
I20260811 22:29:53.803140 15328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f303f87dd74969b7d30defe7579a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33353 } }
I20260811 22:29:53.803377 15328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:53.803414 15328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f303f87dd74969b7d30defe7579a9f, State: Initialized, Role: FOLLOWER
I20260811 22:29:53.803788 15328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f303f87dd74969b7d30defe7579a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33353 } }
I20260811 22:29:53.803989 15328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:53.804039 15328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:53.804112 15328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:53.805044 15328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f303f87dd74969b7d30defe7579a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33353 } }
I20260811 22:29:53.805365 15328 leader_election.cc:302] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72f303f87dd74969b7d30defe7579a9f; no voters:
I20260811 22:29:53.805675 15328 leader_election.cc:288] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:53.805799 15338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:53.806113 15338 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [term 1 LEADER]: Becoming Leader. State: Replica: 72f303f87dd74969b7d30defe7579a9f, State: Running, Role: LEADER
I20260811 22:29:53.806605 15338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f303f87dd74969b7d30defe7579a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33353 } }
I20260811 22:29:53.807101 15328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:53.808426 15339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72f303f87dd74969b7d30defe7579a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f303f87dd74969b7d30defe7579a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33353 } } }
I20260811 22:29:53.808579 15339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:53.809031 15340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72f303f87dd74969b7d30defe7579a9f. Latest consensus state: current_term: 1 leader_uuid: "72f303f87dd74969b7d30defe7579a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f303f87dd74969b7d30defe7579a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33353 } } }
I20260811 22:29:53.809147 15340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:53.809474 15348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:53.812945 15348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:53.818856 15348 catalog_manager.cc:1347] Generated new cluster ID: e8e6b049baee4f78bdd940de36f04e41
I20260811 22:29:53.818994 15348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:53.831019 15348 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:53.831403 15348 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:53.846638 15348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72f303f87dd74969b7d30defe7579a9f: Generated new TSK 0
I20260811 22:29:53.847487 15348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:29:53.850467 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 15248
2026-08-11T22:29:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:53 dist-test-slave-mvvj krb5kdc[15362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:53 dist-test-slave-mvvj krb5kdc[15362](info): set up 2 sockets
Aug 11 22:29:53 dist-test-slave-mvvj krb5kdc[15362](info): commencing operation
krb5kdc: starting...
W20260811 22:29:54.971573 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:55 dist-test-slave-mvvj krb5kdc[15362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:29:55.110715 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:55.227730 15378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:55.228062 15378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:55.228106 15378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:55.232100 15378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:55.232164 15378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:55.232184 15378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:55.232201 15378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:55.232218 15378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:55.237622 15378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:55.238755 15378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:55.239823 15378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:55.248811 15383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:55.248788 15387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:55.248869 15384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:55.249233 15378 server_base.cc:1034] running on GCE node
I20260811 22:29:55.249861 15378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:55.251142 15378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:55.252357 15378 hybrid_clock.cc:648] HybridClock initialized: now 1786487395252330 us; error 49 us; skew 500 ppm
Aug 11 22:29:55 dist-test-slave-mvvj krb5kdc[15362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487395, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:55.255371 15378 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:29:55.256429 15378 webserver.cc:460] Webserver started at http://127.0.0.1:41497/ using document root <none> and password file <none>
I20260811 22:29:55.257058 15378 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:55.257156 15378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:55.257402 15378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:55.259649 15378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3"
format_stamp: "Formatted at 2026-08-11 22:29:55 on dist-test-slave-mvvj"
server_key: "f204570635503882fb86960a22a4edc9"
server_key_iv: "7270df46670b8200485bd40b2e684621"
server_key_version: "encryptionkey@0"
I20260811 22:29:55.260387 15378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3"
format_stamp: "Formatted at 2026-08-11 22:29:55 on dist-test-slave-mvvj"
server_key: "f204570635503882fb86960a22a4edc9"
server_key_iv: "7270df46670b8200485bd40b2e684621"
server_key_version: "encryptionkey@0"
I20260811 22:29:55.264395 15378 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:29:55.266939 15394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:55.268316 15378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:29:55.268465 15378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3"
format_stamp: "Formatted at 2026-08-11 22:29:55 on dist-test-slave-mvvj"
server_key: "f204570635503882fb86960a22a4edc9"
server_key_iv: "7270df46670b8200485bd40b2e684621"
server_key_version: "encryptionkey@0"
I20260811 22:29:55.268586 15378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:55.298708 15378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:55.302081 15378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:55.302325 15378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:55.310930 15378 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41227
I20260811 22:29:55.310951 15451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41227 every 8 connection(s)
I20260811 22:29:55.312027 15378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:55.315156 15452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:55.318826 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 15378
I20260811 22:29:55.318944 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:55.319270 24863 external_mini_cluster.cc:1428] Setting key d82e7d2c1f7a12a8d1acbc20088ec7e3
I20260811 22:29:55.321897 15452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3: Bootstrap starting.
I20260811 22:29:55.325013 15452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:55.325990 15452 log.cc:824] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:55.328737 15452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3: No bootstrap required, opened a new log
Aug 11 22:29:55 dist-test-slave-mvvj krb5kdc[15362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:29:55.333156 15452 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41227 } }
I20260811 22:29:55.333501 15452 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:55.333564 15452 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdcb712cd48f4aeba1ff56ee7161c9f3, State: Initialized, Role: FOLLOWER
I20260811 22:29:55.334107 15452 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41227 } }
I20260811 22:29:55.334288 15452 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:55.334381 15452 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:55.334484 15452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:55.335819 15452 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41227 } }
I20260811 22:29:55.336263 15452 leader_election.cc:302] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdcb712cd48f4aeba1ff56ee7161c9f3; no voters:
I20260811 22:29:55.336701 15452 leader_election.cc:288] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:55.336831 15458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:55.337126 15458 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [term 1 LEADER]: Becoming Leader. State: Replica: cdcb712cd48f4aeba1ff56ee7161c9f3, State: Running, Role: LEADER
I20260811 22:29:55.337522 15458 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41227 } }
I20260811 22:29:55.337797 15456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:55.320889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41629 (local address 127.0.0.1:41227)
0811 22:29:55.321395 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:55.321416 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:55.321463 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:29:55.322375 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:55.322403 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:55.322867 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:55.323293 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:55.324272 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:55.325868 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:55.327159 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:55.327902 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:55.328097 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:55.331183 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:55.331190 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:55.331208 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:55.331243 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:55.334059 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:55.334827 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:55.334836 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:55.334844 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:55.335005 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:55.335307 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:55.335311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:55.335312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:55.335950 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:55.336147 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:55.336430 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:29:55.336771 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":142,"threads_started":1}
I20260811 22:29:55.338091 15452 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:55.339102 15459 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdcb712cd48f4aeba1ff56ee7161c9f3. Latest consensus state: current_term: 1 leader_uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41227 } } }
I20260811 22:29:55.339238 15459 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:55.340126 15460 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcb712cd48f4aeba1ff56ee7161c9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41227 } } }
I20260811 22:29:55.340309 15460 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:55.341739 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 22:29:55.344229 15477 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:29:55.344388 15477 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:29:55.344501 15471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:55.345434 15471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:55.352048 15471 catalog_manager.cc:1347] Generated new cluster ID: ae688f2e2c7a46ca99a844bb89d3d133
I20260811 22:29:55.352126 15471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:55.354193 15456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:55.344275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50043 (local address 127.0.0.1:41227)
0811 22:29:55.344417 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:55.344424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:55.344653 (+ 229us) server_negotiation.cc:407] Connection header received
0811 22:29:55.344822 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:55.344827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:55.344879 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:55.344983 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:55.345904 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:55.346710 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:55.348209 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:55.348559 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:55.348670 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:55.350458 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:55.350462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:55.350466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:29:55.350495 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:55.352458 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:55.352919 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:55.352923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:55.352925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:55.353016 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:29:55.353309 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:29:55.353313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:29:55.353315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:29:55.353501 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:55.353639 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:55.353891 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 22:29:55.354028 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:29:55.366245 15471 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:55.367369 15471 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:55.373253 15471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdcb712cd48f4aeba1ff56ee7161c9f3: Generated new TSK 0
I20260811 22:29:55.374141 15471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:29:55.380636 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 15378
2026-08-11T22:29:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:55 dist-test-slave-mvvj krb5kdc[15491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:55 dist-test-slave-mvvj krb5kdc[15491](info): set up 2 sockets
Aug 11 22:29:55 dist-test-slave-mvvj krb5kdc[15491](info): commencing operation
krb5kdc: starting...
W20260811 22:29:56.515031 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:56 dist-test-slave-mvvj krb5kdc[15491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:29:56.655485 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:56.776137 15508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:56.776482 15508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:56.776582 15508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:56.780748 15508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:56.780840 15508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:56.780879 15508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:56.780937 15508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:56.780977 15508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:56.786399 15508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:56.787557 15508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:56.788688 15508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:56.797330 15515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:56.797335 15514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:56.797521 15508 server_base.cc:1034] running on GCE node
W20260811 22:29:56.797667 15517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:56.798318 15508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:56.799530 15508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:56.800751 15508 hybrid_clock.cc:648] HybridClock initialized: now 1786487396800712 us; error 50 us; skew 500 ppm
Aug 11 22:29:56 dist-test-slave-mvvj krb5kdc[15491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:56.803714 15508 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:29:56.804682 15508 webserver.cc:460] Webserver started at http://127.0.0.1:37823/ using document root <none> and password file <none>
I20260811 22:29:56.805269 15508 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:56.805366 15508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:56.805749 15508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:56.808058 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "f602d28cf4184264bc05e03ea77d4de3"
format_stamp: "Formatted at 2026-08-11 22:29:56 on dist-test-slave-mvvj"
server_key: "d865a765abe708d463865a451adc4b3c"
server_key_iv: "1e0e28100a8ad327543dffe15e64d98b"
server_key_version: "encryptionkey@0"
I20260811 22:29:56.808701 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "f602d28cf4184264bc05e03ea77d4de3"
format_stamp: "Formatted at 2026-08-11 22:29:56 on dist-test-slave-mvvj"
server_key: "d865a765abe708d463865a451adc4b3c"
server_key_iv: "1e0e28100a8ad327543dffe15e64d98b"
server_key_version: "encryptionkey@0"
I20260811 22:29:56.813238 15508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:29:56.816013 15526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:56.817368 15508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:56.817570 15508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "f602d28cf4184264bc05e03ea77d4de3"
format_stamp: "Formatted at 2026-08-11 22:29:56 on dist-test-slave-mvvj"
server_key: "d865a765abe708d463865a451adc4b3c"
server_key_iv: "1e0e28100a8ad327543dffe15e64d98b"
server_key_version: "encryptionkey@0"
I20260811 22:29:56.817700 15508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:56.835160 15508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:56.838901 15508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:56.839149 15508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:56.847506 15508 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44835
I20260811 22:29:56.847519 15585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44835 every 8 connection(s)
I20260811 22:29:56.848600 15508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:56.851542 15586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:56.853628 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 15508
I20260811 22:29:56.853781 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:56.854110 24863 external_mini_cluster.cc:1428] Setting key f24f8d4f81cd22fe49ac706f30f66116
I20260811 22:29:56.858048 15586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3: Bootstrap starting.
I20260811 22:29:56.860683 15586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:56.861501 15586 log.cc:824] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:56.864562 15586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3: No bootstrap required, opened a new log
I20260811 22:29:56.865064 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:56.855659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34045 (local address 127.0.0.1:44835)
0811 22:29:56.856195 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:56.856216 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:56.856259 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:29:56.857108 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:56.857129 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:56.857529 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:56.857919 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:56.858411 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:56.859531 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:56.860718 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:56.861292 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:56.861480 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:56.862725 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:56.862732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:56.862753 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:56.863295 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:56.863392 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:56.863427 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:56.863545 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:56.863733 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 22:29:56.863999 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":182,"threads_started":1}
I20260811 22:29:56.868683 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:29:56.869247 15586 raft_consensus.cc:348] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f602d28cf4184264bc05e03ea77d4de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260811 22:29:56.869540 15586 raft_consensus.cc:374] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:56.869621 15586 raft_consensus.cc:729] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f602d28cf4184264bc05e03ea77d4de3, State: Initialized, Role: FOLLOWER
I20260811 22:29:56.870167 15586 consensus_queue.cc:260] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f602d28cf4184264bc05e03ea77d4de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260811 22:29:56.870467 15586 raft_consensus.cc:388] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:56.870636 15586 raft_consensus.cc:482] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:56.870764 15586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:56.872486 15586 raft_consensus.cc:504] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f602d28cf4184264bc05e03ea77d4de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260811 22:29:56.872923 15586 leader_election.cc:302] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f602d28cf4184264bc05e03ea77d4de3; no voters:
I20260811 22:29:56.873354 15586 leader_election.cc:288] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:56.873512 15596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:56.873792 15596 raft_consensus.cc:686] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [term 1 LEADER]: Becoming Leader. State: Replica: f602d28cf4184264bc05e03ea77d4de3, State: Running, Role: LEADER
I20260811 22:29:56.874187 15596 consensus_queue.cc:237] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f602d28cf4184264bc05e03ea77d4de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260811 22:29:56.874912 15586 sys_catalog.cc:564] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:56.876356 15598 sys_catalog.cc:455] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f602d28cf4184264bc05e03ea77d4de3. Latest consensus state: current_term: 1 leader_uuid: "f602d28cf4184264bc05e03ea77d4de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f602d28cf4184264bc05e03ea77d4de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } } }
I20260811 22:29:56.876503 15598 sys_catalog.cc:458] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:56.876653 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:56.871222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47441 (local address 127.0.0.1:44835)
0811 22:29:56.871379 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:56.871384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:56.871592 (+ 208us) server_negotiation.cc:407] Connection header received
0811 22:29:56.871739 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:56.871742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:56.871787 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:56.871917 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:56.872285 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:56.872900 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:56.874173 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:29:56.874490 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:29:56.874596 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:29:56.875739 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:56.875743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:56.875746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:56.875761 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:56.875795 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:56.875811 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:56.875910 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:56.876439 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 22:29:56.876504 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:29:56.877094 15599 sys_catalog.cc:455] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f602d28cf4184264bc05e03ea77d4de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f602d28cf4184264bc05e03ea77d4de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } } }
I20260811 22:29:56.877252 15599 sys_catalog.cc:458] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3 [sys.catalog]: This master's current role is: LEADER
W20260811 22:29:56.881007 15612 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:29:56.881131 15612 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:29:56.881220 15606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:56.882102 15606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:56.888118 15606 catalog_manager.cc:1347] Generated new cluster ID: af1ca76feb4f405f96c2945ca445f0f2
I20260811 22:29:56.888213 15606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:56.918386 15606 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:56.919450 15606 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:56.924108 15606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f602d28cf4184264bc05e03ea77d4de3: Generated new TSK 0
I20260811 22:29:56.924983 15606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:29:56.942045 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 15508
2026-08-11T22:29:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:57 dist-test-slave-mvvj krb5kdc[15618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:57 dist-test-slave-mvvj krb5kdc[15618](info): set up 2 sockets
Aug 11 22:29:57 dist-test-slave-mvvj krb5kdc[15618](info): commencing operation
krb5kdc: starting...
W20260811 22:29:58.138648 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:58 dist-test-slave-mvvj krb5kdc[15618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:29:58.288486 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:58.409538 15634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:58.409905 15634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:58.409996 15634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:58.414235 15634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:58.414359 15634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:58.414502 15634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:58.414551 15634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:58.414587 15634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:58.420480 15634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:58.421775 15634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:58.422981 15634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:58.432853 15640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:58.432853 15639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:58.432880 15642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:29:58.433205 15634 server_base.cc:1034] running on GCE node
I20260811 22:29:58.433944 15634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:29:58.435334 15634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:29:58.436561 15634 hybrid_clock.cc:648] HybridClock initialized: now 1786487398436490 us; error 60 us; skew 500 ppm
Aug 11 22:29:58 dist-test-slave-mvvj krb5kdc[15618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487398, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:29:58.439682 15634 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:29:58.440692 15634 webserver.cc:460] Webserver started at http://127.0.0.1:38655/ using document root <none> and password file <none>
I20260811 22:29:58.441263 15634 fs_manager.cc:359] Metadata directory not provided
I20260811 22:29:58.441325 15634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:29:58.441506 15634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:29:58.443694 15634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "076deac64d2142a28a7d73384dce57b1"
format_stamp: "Formatted at 2026-08-11 22:29:58 on dist-test-slave-mvvj"
server_key: "abb07b1ce49e759b90aa40eb9248d4c1"
server_key_iv: "f43fbf347e5624d7e3fca0baad8ac1f5"
server_key_version: "encryptionkey@0"
I20260811 22:29:58.444334 15634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "076deac64d2142a28a7d73384dce57b1"
format_stamp: "Formatted at 2026-08-11 22:29:58 on dist-test-slave-mvvj"
server_key: "abb07b1ce49e759b90aa40eb9248d4c1"
server_key_iv: "f43fbf347e5624d7e3fca0baad8ac1f5"
server_key_version: "encryptionkey@0"
I20260811 22:29:58.448382 15634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:29:58.451330 15650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:29:58.452893 15634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:29:58.453054 15634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "076deac64d2142a28a7d73384dce57b1"
format_stamp: "Formatted at 2026-08-11 22:29:58 on dist-test-slave-mvvj"
server_key: "abb07b1ce49e759b90aa40eb9248d4c1"
server_key_iv: "f43fbf347e5624d7e3fca0baad8ac1f5"
server_key_version: "encryptionkey@0"
I20260811 22:29:58.453171 15634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:29:58.458235 15634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:29:58.461760 15634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:29:58.461933 15634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:29:58.469313 15634 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40539
I20260811 22:29:58.469385 15711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40539 every 8 connection(s)
I20260811 22:29:58.470335 15634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:29:58.473475 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:29:58.477051 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 15634
I20260811 22:29:58.477267 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:29:58.477663 24863 external_mini_cluster.cc:1428] Setting key 819a5136ceb45fb1ba806ac1b862feeb
I20260811 22:29:58.481496 15712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1: Bootstrap starting.
I20260811 22:29:58.484337 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:58.479603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42321 (local address 127.0.0.1:40539)
0811 22:29:58.480297 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:58.480315 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:58.480365 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:29:58.481187 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:58.481207 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:58.481547 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:58.481972 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:58.482912 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:58.482917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:58.482930 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:58.482942 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:58.483007 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:58.483028 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:58.483202 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:58.483288 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 22:29:58.483469 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":249,"threads_started":1}
I20260811 22:29:58.484648 15712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1: Neither blocks nor log segments found. Creating new log.
I20260811 22:29:58.485627 15712 log.cc:824] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:29:58.488152 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:29:58.488648 15712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1: No bootstrap required, opened a new log
I20260811 22:29:58.492409 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:29:58.490521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43123 (local address 127.0.0.1:40539)
0811 22:29:58.490658 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:29:58.490663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:29:58.490827 (+ 164us) server_negotiation.cc:407] Connection header received
0811 22:29:58.490975 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:29:58.490977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:29:58.491021 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:29:58.491090 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:29:58.491928 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:29:58.491931 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:29:58.491933 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:29:58.491944 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:29:58.491970 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:29:58.491981 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:29:58.492076 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:29:58.492206 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 22:29:58.492245 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:29:58.493992 15712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076deac64d2142a28a7d73384dce57b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } }
I20260811 22:29:58.494248 15712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:29:58.494334 15712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076deac64d2142a28a7d73384dce57b1, State: Initialized, Role: FOLLOWER
I20260811 22:29:58.494831 15712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076deac64d2142a28a7d73384dce57b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } }
I20260811 22:29:58.494997 15712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:29:58.495069 15712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:29:58.495195 15712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:29:58.496345 15712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076deac64d2142a28a7d73384dce57b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } }
I20260811 22:29:58.496764 15712 leader_election.cc:302] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 076deac64d2142a28a7d73384dce57b1; no voters:
I20260811 22:29:58.497161 15712 leader_election.cc:288] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:29:58.497323 15724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:29:58.497613 15724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [term 1 LEADER]: Becoming Leader. State: Replica: 076deac64d2142a28a7d73384dce57b1, State: Running, Role: LEADER
I20260811 22:29:58.497991 15724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076deac64d2142a28a7d73384dce57b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } }
I20260811 22:29:58.498567 15712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:29:58.500069 15725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "076deac64d2142a28a7d73384dce57b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076deac64d2142a28a7d73384dce57b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } } }
I20260811 22:29:58.500216 15725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:29:58.500941 15726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 076deac64d2142a28a7d73384dce57b1. Latest consensus state: current_term: 1 leader_uuid: "076deac64d2142a28a7d73384dce57b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076deac64d2142a28a7d73384dce57b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } } }
I20260811 22:29:58.501044 15726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1 [sys.catalog]: This master's current role is: LEADER
W20260811 22:29:58.503953 15740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:29:58.504024 15740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:29:58.504105 15735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:29:58.504842 15735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:29:58.512485 15735 catalog_manager.cc:1347] Generated new cluster ID: 16cf5b5ee55c49bb83c7927a50bc83d1
I20260811 22:29:58.512557 15735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:29:58.541601 15735 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:29:58.541985 15735 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:29:58.550734 15735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 076deac64d2142a28a7d73384dce57b1: Generated new TSK 0
I20260811 22:29:58.551541 15735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:29:58.563782 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 15634
2026-08-11T22:29:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1623 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:29:58 dist-test-slave-mvvj krb5kdc[15746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:29:58 dist-test-slave-mvvj krb5kdc[15746](info): set up 2 sockets
Aug 11 22:29:58 dist-test-slave-mvvj krb5kdc[15746](info): commencing operation
krb5kdc: starting...
W20260811 22:29:59.711009 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:29:59 dist-test-slave-mvvj krb5kdc[15746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:29:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:29:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:29:59.855223 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:29:59.977244 15760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:29:59.977605 15760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:29:59.977691 15760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:29:59.981982 15760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:29:59.982098 15760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:29:59.982136 15760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:29:59.982175 15760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:29:59.982208 15760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:29:59.988066 15760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:29:59.989343 15760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:29:59.990517 15760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:29:59.999847 15767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:00.000077 15760 server_base.cc:1034] running on GCE node
W20260811 22:29:59.999835 15766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:29:59.999970 15769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:00.001694 15760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:00.003130 15760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:00.004376 15760 hybrid_clock.cc:648] HybridClock initialized: now 1786487400004344 us; error 52 us; skew 500 ppm
Aug 11 22:30:00 dist-test-slave-mvvj krb5kdc[15746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487400, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:00.012876 15760 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:30:00.014081 15760 webserver.cc:460] Webserver started at http://127.0.0.1:37479/ using document root <none> and password file <none>
I20260811 22:30:00.014740 15760 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:00.014842 15760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:00.015105 15760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:00.017522 15760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "9d7d85954dc0473e8d5dd4573f092131"
format_stamp: "Formatted at 2026-08-11 22:30:00 on dist-test-slave-mvvj"
server_key: "dffdf7f49f29a5ba9cecb669b5297dc3"
server_key_iv: "60b8f3f760ad655cfd5caf7940e22ece"
server_key_version: "encryptionkey@0"
I20260811 22:30:00.018154 15760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "9d7d85954dc0473e8d5dd4573f092131"
format_stamp: "Formatted at 2026-08-11 22:30:00 on dist-test-slave-mvvj"
server_key: "dffdf7f49f29a5ba9cecb669b5297dc3"
server_key_iv: "60b8f3f760ad655cfd5caf7940e22ece"
server_key_version: "encryptionkey@0"
I20260811 22:30:00.022477 15760 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:00.025687 15776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:00.027281 15760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:30:00.027453 15760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "9d7d85954dc0473e8d5dd4573f092131"
format_stamp: "Formatted at 2026-08-11 22:30:00 on dist-test-slave-mvvj"
server_key: "dffdf7f49f29a5ba9cecb669b5297dc3"
server_key_iv: "60b8f3f760ad655cfd5caf7940e22ece"
server_key_version: "encryptionkey@0"
I20260811 22:30:00.027597 15760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:00.044823 15760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:00.048377 15760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:00.048666 15760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:00.057495 15760 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36645
I20260811 22:30:00.057538 15837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36645 every 8 connection(s)
I20260811 22:30:00.058609 15760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:00.062254 15838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:00.064515 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 15760
I20260811 22:30:00.064688 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:00.065047 24863 external_mini_cluster.cc:1428] Setting key f5d7dddeb5038f90b6c69c439f0357e9
I20260811 22:30:00.070283 15838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131: Bootstrap starting.
I20260811 22:30:00.073194 15838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:00.074265 15838 log.cc:824] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:30:00 dist-test-slave-mvvj krb5kdc[15746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:30:00.077158 15838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131: No bootstrap required, opened a new log
I20260811 22:30:00.081863 15838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7d85954dc0473e8d5dd4573f092131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36645 } }
I20260811 22:30:00.082180 15838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:00.082266 15838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7d85954dc0473e8d5dd4573f092131, State: Initialized, Role: FOLLOWER
I20260811 22:30:00.082551 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:00.066848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:40777 (local address 127.0.0.1:36645)
0811 22:30:00.067398 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:00.067415 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:00.067465 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:30:00.068265 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:00.068286 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:00.068682 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:00.069141 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:00.070418 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:00.071532 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:00.072590 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:00.073106 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:00.073302 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:00.076752 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:00.076757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:00.076773 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:00.076803 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:00.078771 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:00.079600 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:00.079606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:00.079612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:00.079741 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:00.080167 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:00.080170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:00.080171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:00.080648 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:00.080832 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:00.081216 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 22:30:00.081555 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":174,"threads_started":1}
I20260811 22:30:00.082792 15838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7d85954dc0473e8d5dd4573f092131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36645 } }
I20260811 22:30:00.082952 15838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:00.083076 15838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:00.083212 15838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:00.084369 15838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7d85954dc0473e8d5dd4573f092131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36645 } }
I20260811 22:30:00.084793 15838 leader_election.cc:302] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d7d85954dc0473e8d5dd4573f092131; no voters:
I20260811 22:30:00.085175 15838 leader_election.cc:288] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:00.085332 15844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:00.085629 15844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [term 1 LEADER]: Becoming Leader. State: Replica: 9d7d85954dc0473e8d5dd4573f092131, State: Running, Role: LEADER
I20260811 22:30:00.085992 15844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7d85954dc0473e8d5dd4573f092131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36645 } }
I20260811 22:30:00.086220 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:00.086683 15838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:00.089280 15846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d7d85954dc0473e8d5dd4573f092131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7d85954dc0473e8d5dd4573f092131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36645 } } }
I20260811 22:30:00.090072 15846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:00.089676 15847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d7d85954dc0473e8d5dd4573f092131. Latest consensus state: current_term: 1 leader_uuid: "9d7d85954dc0473e8d5dd4573f092131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7d85954dc0473e8d5dd4573f092131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36645 } } }
I20260811 22:30:00.090376 15847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131 [sys.catalog]: This master's current role is: LEADER
W20260811 22:30:00.093532 15864 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:00.093634 15864 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:00.093713 15867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:00.094713 15867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:00.100457 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:00.089156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:43029 (local address 127.0.0.1:36645)
0811 22:30:00.089522 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:00.089527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:00.090182 (+ 655us) server_negotiation.cc:407] Connection header received
0811 22:30:00.090389 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:00.090394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:00.090461 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:00.090555 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:00.091427 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:00.092298 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:00.093541 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:00.093938 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:00.094033 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:00.095942 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:00.095947 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:00.095951 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:00.095984 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:00.098070 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:00.098754 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:00.098759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:00.098761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:00.098882 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:00.099333 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:00.099337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:00.099338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:00.099548 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:00.099712 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:00.100160 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 22:30:00.100283 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260811 22:30:00.101606 15867 catalog_manager.cc:1347] Generated new cluster ID: ccfa593fbf4d447d8226708792f26944
I20260811 22:30:00.102035 15867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:00.122903 15867 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:00.124233 15867 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:00.131330 15867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d7d85954dc0473e8d5dd4573f092131: Generated new TSK 0
I20260811 22:30:00.132920 15867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:00.143982 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 15760
W20260811 22:30:00.153537 15744 connection.cc:535] client connection to 127.0.0.1:36645 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36645): Connection reset by peer (error 104)
2026-08-11T22:30:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:00 dist-test-slave-mvvj krb5kdc[15873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:00 dist-test-slave-mvvj krb5kdc[15873](info): set up 2 sockets
Aug 11 22:30:00 dist-test-slave-mvvj krb5kdc[15873](info): commencing operation
krb5kdc: starting...
W20260811 22:30:01.284228 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:01 dist-test-slave-mvvj krb5kdc[15873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:30:01.435470 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:01.559991 15890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:01.560365 15890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:01.560467 15890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:01.565179 15890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:01.565291 15890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:01.565330 15890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:01.565387 15890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:01.565438 15890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:01.571245 15890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:01.572614 15890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:01.573807 15890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:01.583122 15895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:01.583158 15898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:01.583194 15896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:01.583420 15890 server_base.cc:1034] running on GCE node
I20260811 22:30:01.584276 15890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:01.585469 15890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:01.586681 15890 hybrid_clock.cc:648] HybridClock initialized: now 1786487401586653 us; error 42 us; skew 500 ppm
Aug 11 22:30:01 dist-test-slave-mvvj krb5kdc[15873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487401, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:01.589797 15890 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:30:01.590838 15890 webserver.cc:460] Webserver started at http://127.0.0.1:39461/ using document root <none> and password file <none>
I20260811 22:30:01.591408 15890 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:01.591470 15890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:01.591652 15890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:01.593899 15890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "9c3588614cc94e4bb2e772834c493f88"
format_stamp: "Formatted at 2026-08-11 22:30:01 on dist-test-slave-mvvj"
server_key: "880087d8d7f7eb4142a133955453c22d"
server_key_iv: "6af048d26eb72d533b8c5084529937c6"
server_key_version: "encryptionkey@0"
I20260811 22:30:01.594468 15890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "9c3588614cc94e4bb2e772834c493f88"
format_stamp: "Formatted at 2026-08-11 22:30:01 on dist-test-slave-mvvj"
server_key: "880087d8d7f7eb4142a133955453c22d"
server_key_iv: "6af048d26eb72d533b8c5084529937c6"
server_key_version: "encryptionkey@0"
I20260811 22:30:01.598495 15890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:30:01.601255 15906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:01.602739 15890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:30:01.602859 15890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "9c3588614cc94e4bb2e772834c493f88"
format_stamp: "Formatted at 2026-08-11 22:30:01 on dist-test-slave-mvvj"
server_key: "880087d8d7f7eb4142a133955453c22d"
server_key_iv: "6af048d26eb72d533b8c5084529937c6"
server_key_version: "encryptionkey@0"
I20260811 22:30:01.603024 15890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:01.619167 15890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:01.622757 15890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:01.622993 15890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:01.631142 15890 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38333
I20260811 22:30:01.631228 15966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38333 every 8 connection(s)
I20260811 22:30:01.632248 15890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:01.634085 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 15890
I20260811 22:30:01.634197 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:01.634513 24863 external_mini_cluster.cc:1428] Setting key a22aadf2fdddc16b688b19bf7e79e807
I20260811 22:30:01.635902 15967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:01.644266 15967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88: Bootstrap starting.
I20260811 22:30:01.646221 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:01.636115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:41083 (local address 127.0.0.1:38333)
0811 22:30:01.636574 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:01.636596 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:01.636642 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:30:01.637639 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:01.637665 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:01.638175 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:01.638616 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:01.639101 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:01.640847 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:01.641898 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:01.642703 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:01.642900 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:01.643926 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:01.643935 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:01.643954 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:01.644524 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:01.644609 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:01.644643 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:01.644796 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:01.645004 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 22:30:01.645160 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":117,"threads_started":1}
I20260811 22:30:01.647344 15967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:01.648332 15967 log.cc:824] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:01.649808 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:01.650832 15967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88: No bootstrap required, opened a new log
I20260811 22:30:01.655064 15967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3588614cc94e4bb2e772834c493f88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } }
I20260811 22:30:01.655392 15967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:01.655484 15967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3588614cc94e4bb2e772834c493f88, State: Initialized, Role: FOLLOWER
I20260811 22:30:01.656016 15967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3588614cc94e4bb2e772834c493f88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } }
I20260811 22:30:01.656198 15967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:01.656284 15967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:01.656418 15967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:01.656921 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:01.651986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:51621 (local address 127.0.0.1:38333)
0811 22:30:01.652119 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:01.652126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:01.652318 (+ 192us) server_negotiation.cc:407] Connection header received
0811 22:30:01.652469 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:01.652471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:01.652542 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:01.652609 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:01.652940 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:01.653872 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:01.655131 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:01.655352 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:01.655477 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:01.656443 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:01.656447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:01.656449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:01.656461 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:01.656488 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:01.656499 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:01.656616 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:01.656742 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 22:30:01.656781 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:30:01.657789 15967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3588614cc94e4bb2e772834c493f88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } }
I20260811 22:30:01.658210 15967 leader_election.cc:302] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c3588614cc94e4bb2e772834c493f88; no voters:
I20260811 22:30:01.658597 15967 leader_election.cc:288] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:01.658672 15980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:01.658965 15980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [term 1 LEADER]: Becoming Leader. State: Replica: 9c3588614cc94e4bb2e772834c493f88, State: Running, Role: LEADER
I20260811 22:30:01.659276 15980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3588614cc94e4bb2e772834c493f88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } }
I20260811 22:30:01.660104 15967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:01.661116 15981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c3588614cc94e4bb2e772834c493f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3588614cc94e4bb2e772834c493f88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } } }
I20260811 22:30:01.661279 15981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:01.661710 15989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:01.662572 15982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c3588614cc94e4bb2e772834c493f88. Latest consensus state: current_term: 1 leader_uuid: "9c3588614cc94e4bb2e772834c493f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3588614cc94e4bb2e772834c493f88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } } }
I20260811 22:30:01.662693 15982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:01.665046 15989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:01.671801 15989 catalog_manager.cc:1347] Generated new cluster ID: 10ca7cba2b6b4b2c9d7e79683dab7e08
I20260811 22:30:01.672031 15989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:01.684852 15989 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:01.685896 15989 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:01.693527 15989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c3588614cc94e4bb2e772834c493f88: Generated new TSK 0
I20260811 22:30:01.694350 15989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:01.703533 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 15890
W20260811 22:30:01.712884 15871 connection.cc:535] client connection to 127.0.0.1:38333 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38333): Connection reset by peer (error 104)
2026-08-11T22:30:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:01 dist-test-slave-mvvj krb5kdc[16001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:01 dist-test-slave-mvvj krb5kdc[16001](info): set up 2 sockets
Aug 11 22:30:01 dist-test-slave-mvvj krb5kdc[16001](info): commencing operation
krb5kdc: starting...
W20260811 22:30:02.885615 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:02 dist-test-slave-mvvj krb5kdc[16001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:30:03.028329 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38557
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:03.149670 16015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:03.149995 16015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:03.150048 16015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:03.154421 16015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:03.154515 16015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:03.154536 16015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:03.154552 16015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:03.154568 16015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:03.160516 16015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:03.161842 16015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:03.162992 16015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:30:03.172152 16015 server_base.cc:1034] running on GCE node
W20260811 22:30:03.172137 16020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:03.172116 16021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:03.172139 16023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:03.173233 16015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:03.174530 16015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:03.175773 16015 hybrid_clock.cc:648] HybridClock initialized: now 1786487403175731 us; error 51 us; skew 500 ppm
Aug 11 22:30:03 dist-test-slave-mvvj krb5kdc[16001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487403, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:03.179059 16015 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:30:03.180231 16015 webserver.cc:460] Webserver started at http://127.0.0.1:33923/ using document root <none> and password file <none>
I20260811 22:30:03.180943 16015 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:03.181052 16015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:03.181331 16015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:03.183768 16015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "4abf027b9a304293b7f30dff92355761"
format_stamp: "Formatted at 2026-08-11 22:30:03 on dist-test-slave-mvvj"
server_key: "efdfd039f405f27a6e615c1a99663407"
server_key_iv: "94e3112685696df3571f57ffecb439a0"
server_key_version: "encryptionkey@0"
I20260811 22:30:03.184530 16015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "4abf027b9a304293b7f30dff92355761"
format_stamp: "Formatted at 2026-08-11 22:30:03 on dist-test-slave-mvvj"
server_key: "efdfd039f405f27a6e615c1a99663407"
server_key_iv: "94e3112685696df3571f57ffecb439a0"
server_key_version: "encryptionkey@0"
I20260811 22:30:03.189101 16015 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:30:03.191778 16030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:03.193163 16015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:03.193339 16015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "4abf027b9a304293b7f30dff92355761"
format_stamp: "Formatted at 2026-08-11 22:30:03 on dist-test-slave-mvvj"
server_key: "efdfd039f405f27a6e615c1a99663407"
server_key_iv: "94e3112685696df3571f57ffecb439a0"
server_key_version: "encryptionkey@0"
I20260811 22:30:03.193476 16015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:03.198597 16015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:03.202250 16015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:03.202618 16015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:03.211619 16015 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38557
I20260811 22:30:03.211650 16089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38557 every 8 connection(s)
I20260811 22:30:03.212814 16015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:03.216462 16091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:03.216722 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 16015
I20260811 22:30:03.216900 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:03.217228 24863 external_mini_cluster.cc:1428] Setting key c5f5fa13de2fd850444b7630b34c1e2d
I20260811 22:30:03.223964 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:03.218978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:43683 (local address 127.0.0.1:38557)
0811 22:30:03.219480 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:03.219508 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:03.219558 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:30:03.220432 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:03.220459 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:03.220875 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:03.221319 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:03.222134 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:03.222140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:03.222157 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:03.222171 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:03.222240 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:03.222270 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:03.222439 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:03.222526 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 22:30:03.222707 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":137,"threads_started":1}
I20260811 22:30:03.225128 16091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761: Bootstrap starting.
I20260811 22:30:03.227356 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:03.227797 16091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:03.228860 16091 log.cc:824] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:03.231743 16091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761: No bootstrap required, opened a new log
I20260811 22:30:03.232038 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:03.230006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:56111 (local address 127.0.0.1:38557)
0811 22:30:03.230150 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:03.230155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:03.230374 (+ 219us) server_negotiation.cc:407] Connection header received
0811 22:30:03.230557 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:03.230559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:03.230592 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:03.230664 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:03.231490 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:03.231494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:03.231496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:03.231535 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:03.231570 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:03.231581 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:03.231670 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:03.231786 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 22:30:03.231903 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:30:03.235713 16091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4abf027b9a304293b7f30dff92355761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38557 } }
I20260811 22:30:03.236124 16091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:03.236212 16091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abf027b9a304293b7f30dff92355761, State: Initialized, Role: FOLLOWER
I20260811 22:30:03.236665 16091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4abf027b9a304293b7f30dff92355761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38557 } }
I20260811 22:30:03.236822 16091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:03.236910 16091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:03.237025 16091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:03.238122 16091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4abf027b9a304293b7f30dff92355761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38557 } }
I20260811 22:30:03.238538 16091 leader_election.cc:302] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4abf027b9a304293b7f30dff92355761; no voters:
I20260811 22:30:03.238916 16091 leader_election.cc:288] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:03.239030 16103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:03.239324 16103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [term 1 LEADER]: Becoming Leader. State: Replica: 4abf027b9a304293b7f30dff92355761, State: Running, Role: LEADER
I20260811 22:30:03.239686 16103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4abf027b9a304293b7f30dff92355761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38557 } }
I20260811 22:30:03.240396 16091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:03.241670 16104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4abf027b9a304293b7f30dff92355761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4abf027b9a304293b7f30dff92355761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38557 } } }
I20260811 22:30:03.241820 16104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:03.242146 16105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4abf027b9a304293b7f30dff92355761. Latest consensus state: current_term: 1 leader_uuid: "4abf027b9a304293b7f30dff92355761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4abf027b9a304293b7f30dff92355761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38557 } } }
I20260811 22:30:03.242249 16105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:03.242378 16114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:03.245488 16114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:03.251559 16114 catalog_manager.cc:1347] Generated new cluster ID: 23f49a3228cb467493e74ecd49816e48
I20260811 22:30:03.251660 16114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:03.272418 16114 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:03.272832 16114 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:03.282222 16114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4abf027b9a304293b7f30dff92355761: Generated new TSK 0
I20260811 22:30:03.283077 16114 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:03.292539 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 16015
2026-08-11T22:30:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1588 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:03 dist-test-slave-mvvj krb5kdc[16129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:03 dist-test-slave-mvvj krb5kdc[16129](info): set up 2 sockets
Aug 11 22:30:03 dist-test-slave-mvvj krb5kdc[16129](info): commencing operation
krb5kdc: starting...
W20260811 22:30:04.426247 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:04 dist-test-slave-mvvj krb5kdc[16129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:30:04.567935 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:04.702296 16143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:04.702669 16143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:04.702772 16143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:04.707098 16143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:04.707214 16143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:04.707255 16143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:04.707315 16143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:04.707355 16143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:04.713197 16143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43511
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:04.714478 16143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:04.715636 16143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:04.724934 16149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:04.724972 16148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:04.724969 16151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:04.725301 16143 server_base.cc:1034] running on GCE node
I20260811 22:30:04.726119 16143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:04.727774 16143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:04.729044 16143 hybrid_clock.cc:648] HybridClock initialized: now 1786487404729013 us; error 40 us; skew 500 ppm
Aug 11 22:30:04 dist-test-slave-mvvj krb5kdc[16129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487404, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:04.733035 16143 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:30:04.734139 16143 webserver.cc:460] Webserver started at http://127.0.0.1:44985/ using document root <none> and password file <none>
I20260811 22:30:04.734776 16143 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:04.734848 16143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:04.735082 16143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:04.737475 16143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "b1a85ee07c2d4e679764a71cd4c0e743"
format_stamp: "Formatted at 2026-08-11 22:30:04 on dist-test-slave-mvvj"
server_key: "0ad6fa180555797533a482f5611fb6a5"
server_key_iv: "4e710d9c19ed250592ad49d2cba00c5e"
server_key_version: "encryptionkey@0"
I20260811 22:30:04.738106 16143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "b1a85ee07c2d4e679764a71cd4c0e743"
format_stamp: "Formatted at 2026-08-11 22:30:04 on dist-test-slave-mvvj"
server_key: "0ad6fa180555797533a482f5611fb6a5"
server_key_iv: "4e710d9c19ed250592ad49d2cba00c5e"
server_key_version: "encryptionkey@0"
I20260811 22:30:04.742532 16143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:30:04.745389 16158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:04.746840 16143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:30:04.746985 16143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "b1a85ee07c2d4e679764a71cd4c0e743"
format_stamp: "Formatted at 2026-08-11 22:30:04 on dist-test-slave-mvvj"
server_key: "0ad6fa180555797533a482f5611fb6a5"
server_key_iv: "4e710d9c19ed250592ad49d2cba00c5e"
server_key_version: "encryptionkey@0"
I20260811 22:30:04.747113 16143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:04.757607 16143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:04.761263 16143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:04.761497 16143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:04.769038 16143 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43511
I20260811 22:30:04.769063 16219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43511 every 8 connection(s)
I20260811 22:30:04.770081 16143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:04.773191 16220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:04.776587 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 16143
I20260811 22:30:04.776710 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:04.777022 24863 external_mini_cluster.cc:1428] Setting key 20fcd0322f7f535f198ea8df4b359c8f
I20260811 22:30:04.780019 16220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743: Bootstrap starting.
I20260811 22:30:04.782964 16220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:04.783813 16220 log.cc:824] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:04.786285 16220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743: No bootstrap required, opened a new log
Aug 11 22:30:04 dist-test-slave-mvvj krb5kdc[16129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:30:04.790841 16220 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a85ee07c2d4e679764a71cd4c0e743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } }
I20260811 22:30:04.791172 16220 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:04.791251 16220 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a85ee07c2d4e679764a71cd4c0e743, State: Initialized, Role: FOLLOWER
I20260811 22:30:04.791806 16220 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a85ee07c2d4e679764a71cd4c0e743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } }
I20260811 22:30:04.792011 16220 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:04.792107 16220 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:04.792212 16220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:04.793246 16220 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a85ee07c2d4e679764a71cd4c0e743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } }
I20260811 22:30:04.793654 16220 leader_election.cc:302] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1a85ee07c2d4e679764a71cd4c0e743; no voters:
I20260811 22:30:04.794044 16220 leader_election.cc:288] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:04.794148 16225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:04.794409 16225 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [term 1 LEADER]: Becoming Leader. State: Replica: b1a85ee07c2d4e679764a71cd4c0e743, State: Running, Role: LEADER
I20260811 22:30:04.794538 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:04.778629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:46747 (local address 127.0.0.1:43511)
0811 22:30:04.779117 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:04.779143 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:04.779188 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:30:04.780100 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:04.780127 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:04.780593 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:04.780969 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:04.781646 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:04.783125 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:04.784147 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:04.784769 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:04.784939 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:04.788015 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:04.788022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:04.788042 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:04.788076 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:04.790852 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:04.791557 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:04.791565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:04.791571 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:04.791717 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:04.792166 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:04.792169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:04.792170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:04.792675 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:04.792932 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:04.793263 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 22:30:04.793517 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":138,"threads_started":1}
I20260811 22:30:04.794747 16225 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a85ee07c2d4e679764a71cd4c0e743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } }
I20260811 22:30:04.795446 16220 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:04.796047 16227 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1a85ee07c2d4e679764a71cd4c0e743. Latest consensus state: current_term: 1 leader_uuid: "b1a85ee07c2d4e679764a71cd4c0e743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a85ee07c2d4e679764a71cd4c0e743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } } }
I20260811 22:30:04.796170 16227 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:04.796936 16226 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1a85ee07c2d4e679764a71cd4c0e743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a85ee07c2d4e679764a71cd4c0e743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43511 } } }
I20260811 22:30:04.797068 16226 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:04.797456 16230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:04.798491 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:04.800865 16230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:04.808017 16230 catalog_manager.cc:1347] Generated new cluster ID: b4845818821744bb96b5cec660ee8a5d
I20260811 22:30:04.808106 16230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:04.810460 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:04.800658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:37647 (local address 127.0.0.1:43511)
0811 22:30:04.800778 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:04.800783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:04.801118 (+ 335us) server_negotiation.cc:407] Connection header received
0811 22:30:04.801268 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:04.801271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:04.801324 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:04.801421 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:04.802139 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:04.802671 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:04.803791 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:04.804238 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:04.804356 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:04.806142 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:04.806148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:04.806151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:04.806182 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:04.808263 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:04.808930 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:04.808934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:04.808936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:04.809045 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:04.809437 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:04.809441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:04.809443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:04.809694 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:04.809888 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:04.810159 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 22:30:04.810278 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:30:04.842423 16230 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:04.843977 16230 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:04.850878 16230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1a85ee07c2d4e679764a71cd4c0e743: Generated new TSK 0
I20260811 22:30:04.851917 16230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:04.861652 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 16143
W20260811 22:30:04.871122 16127 connection.cc:535] client connection to 127.0.0.1:43511 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43511): Connection reset by peer (error 104)
2026-08-11T22:30:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:04 dist-test-slave-mvvj krb5kdc[16252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:04 dist-test-slave-mvvj krb5kdc[16252](info): set up 2 sockets
Aug 11 22:30:04 dist-test-slave-mvvj krb5kdc[16252](info): commencing operation
krb5kdc: starting...
W20260811 22:30:05.980847 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:06 dist-test-slave-mvvj krb5kdc[16252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:30:06.128429 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40401
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:06.245288 16265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:06.245652 16265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:06.245752 16265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:06.249939 16265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:06.250051 16265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:06.250090 16265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:06.250131 16265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:06.250168 16265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:06.255815 16265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40401
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:06.257162 16265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:06.258262 16265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:06.267457 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:06.267485 16273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:06.267544 16275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:06.267864 16265 server_base.cc:1034] running on GCE node
I20260811 22:30:06.268504 16265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:06.269745 16265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:06.270970 16265 hybrid_clock.cc:648] HybridClock initialized: now 1786487406270943 us; error 39 us; skew 500 ppm
Aug 11 22:30:06 dist-test-slave-mvvj krb5kdc[16252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487406, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:06.274230 16265 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:30:06.275285 16265 webserver.cc:460] Webserver started at http://127.0.0.1:38863/ using document root <none> and password file <none>
I20260811 22:30:06.275975 16265 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:06.276077 16265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:06.276335 16265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:06.278752 16265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "33c49734ad194ddf9cbf431450eb74c8"
format_stamp: "Formatted at 2026-08-11 22:30:06 on dist-test-slave-mvvj"
server_key: "affdfef751f46f455bfb49e8560434aa"
server_key_iv: "f1b2ee03e1e3211dae3f9ec362128f7c"
server_key_version: "encryptionkey@0"
I20260811 22:30:06.279404 16265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "33c49734ad194ddf9cbf431450eb74c8"
format_stamp: "Formatted at 2026-08-11 22:30:06 on dist-test-slave-mvvj"
server_key: "affdfef751f46f455bfb49e8560434aa"
server_key_iv: "f1b2ee03e1e3211dae3f9ec362128f7c"
server_key_version: "encryptionkey@0"
I20260811 22:30:06.283550 16265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:30:06.286303 16284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:06.287701 16265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:30:06.287920 16265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "33c49734ad194ddf9cbf431450eb74c8"
format_stamp: "Formatted at 2026-08-11 22:30:06 on dist-test-slave-mvvj"
server_key: "affdfef751f46f455bfb49e8560434aa"
server_key_iv: "f1b2ee03e1e3211dae3f9ec362128f7c"
server_key_version: "encryptionkey@0"
I20260811 22:30:06.288061 16265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:06.315737 16265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:06.319789 16265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:06.320073 16265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:06.327720 16265 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40401
I20260811 22:30:06.327749 16345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40401 every 8 connection(s)
I20260811 22:30:06.328743 16265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:06.331701 16346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:06.337702 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 16265
I20260811 22:30:06.337703 16346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8: Bootstrap starting.
I20260811 22:30:06.337824 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:06.338114 24863 external_mini_cluster.cc:1428] Setting key 85d7d4dd7bde456f71d163c27c2e1e80
I20260811 22:30:06.340714 16346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:06.341653 16346 log.cc:824] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:06.344118 16346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8: No bootstrap required, opened a new log
I20260811 22:30:06.348754 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:06.339613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:58577 (local address 127.0.0.1:40401)
0811 22:30:06.340141 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:06.340161 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:06.340205 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:30:06.341170 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:06.341196 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:06.341669 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:06.342064 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:06.342422 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:06.343852 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:06.344795 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:06.345460 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:06.345628 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:06.346492 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:06.346498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:06.346517 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:06.347094 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:06.347178 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:06.347214 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:06.347376 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:06.347550 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:30:06.347689 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":199,"threads_started":1}
I20260811 22:30:06.348730 16346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33c49734ad194ddf9cbf431450eb74c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40401 } }
I20260811 22:30:06.349037 16346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:06.349089 16346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33c49734ad194ddf9cbf431450eb74c8, State: Initialized, Role: FOLLOWER
I20260811 22:30:06.349555 16346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33c49734ad194ddf9cbf431450eb74c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40401 } }
I20260811 22:30:06.349676 16346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:06.349717 16346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:06.349812 16346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:06.351029 16346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33c49734ad194ddf9cbf431450eb74c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40401 } }
I20260811 22:30:06.351400 16346 leader_election.cc:302] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33c49734ad194ddf9cbf431450eb74c8; no voters:
I20260811 22:30:06.351768 16346 leader_election.cc:288] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:06.351915 16351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:06.352152 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:06.352186 16351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [term 1 LEADER]: Becoming Leader. State: Replica: 33c49734ad194ddf9cbf431450eb74c8, State: Running, Role: LEADER
I20260811 22:30:06.352507 16351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33c49734ad194ddf9cbf431450eb74c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40401 } }
I20260811 22:30:06.353117 16346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:06.353832 16351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33c49734ad194ddf9cbf431450eb74c8. Latest consensus state: current_term: 1 leader_uuid: "33c49734ad194ddf9cbf431450eb74c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33c49734ad194ddf9cbf431450eb74c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40401 } } }
I20260811 22:30:06.353957 16351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:06.354365 16354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33c49734ad194ddf9cbf431450eb74c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33c49734ad194ddf9cbf431450eb74c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40401 } } }
I20260811 22:30:06.354475 16354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:06.354590 16363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:06.358085 16363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:06.360905 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:06.354637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:56085 (local address 127.0.0.1:40401)
0811 22:30:06.354770 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:06.354775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:06.355472 (+ 697us) server_negotiation.cc:407] Connection header received
0811 22:30:06.355724 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:06.355731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:06.355790 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:06.355895 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:06.356398 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:06.357414 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:06.358655 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:06.358996 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:06.359074 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:06.360349 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:06.360356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:06.360359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:06.360376 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:06.360410 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:06.360424 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:06.360538 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:06.360714 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:30:06.360757 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:30:06.364668 16363 catalog_manager.cc:1347] Generated new cluster ID: 06565ee1e0dd4c20b8e83fcc2face95b
I20260811 22:30:06.364755 16363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:06.381070 16363 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:06.382470 16363 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:06.390566 16363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33c49734ad194ddf9cbf431450eb74c8: Generated new TSK 0
I20260811 22:30:06.391534 16363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:06.397584 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 16265
W20260811 22:30:06.407442 16250 connection.cc:535] client connection to 127.0.0.1:40401 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40401): Connection reset by peer (error 104)
2026-08-11T22:30:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:06 dist-test-slave-mvvj krb5kdc[16383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:06 dist-test-slave-mvvj krb5kdc[16383](info): set up 2 sockets
Aug 11 22:30:06 dist-test-slave-mvvj krb5kdc[16383](info): commencing operation
krb5kdc: starting...
W20260811 22:30:07.555114 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:07 dist-test-slave-mvvj krb5kdc[16383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:30:07.700507 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35893
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:07.822949 16397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:07.823318 16397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:07.823410 16397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:07.827699 16397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:07.827821 16397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:07.827884 16397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:07.827946 16397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:07.827987 16397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:07.833730 16397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35893
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:07.834995 16397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:07.836169 16397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:07.845750 16406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:07.845921 16397 server_base.cc:1034] running on GCE node
W20260811 22:30:07.845825 16404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:07.846164 16403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:07.846931 16397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:07.848381 16397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:07.849581 16397 hybrid_clock.cc:648] HybridClock initialized: now 1786487407849546 us; error 49 us; skew 500 ppm
Aug 11 22:30:07 dist-test-slave-mvvj krb5kdc[16383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487407, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:07.852627 16397 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:30:07.853674 16397 webserver.cc:460] Webserver started at http://127.0.0.1:42273/ using document root <none> and password file <none>
I20260811 22:30:07.854280 16397 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:07.854382 16397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:07.854627 16397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:07.856875 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "b521460ae73648a8802a0a2eff324aca"
format_stamp: "Formatted at 2026-08-11 22:30:07 on dist-test-slave-mvvj"
server_key: "f9186754224ea20f0324f7c68135afea"
server_key_iv: "b04e9521d3d78d7f0ce7e72e73e13881"
server_key_version: "encryptionkey@0"
I20260811 22:30:07.857513 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "b521460ae73648a8802a0a2eff324aca"
format_stamp: "Formatted at 2026-08-11 22:30:07 on dist-test-slave-mvvj"
server_key: "f9186754224ea20f0324f7c68135afea"
server_key_iv: "b04e9521d3d78d7f0ce7e72e73e13881"
server_key_version: "encryptionkey@0"
I20260811 22:30:07.861707 16397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:07.864619 16415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:07.866236 16397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:07.866425 16397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "b521460ae73648a8802a0a2eff324aca"
format_stamp: "Formatted at 2026-08-11 22:30:07 on dist-test-slave-mvvj"
server_key: "f9186754224ea20f0324f7c68135afea"
server_key_iv: "b04e9521d3d78d7f0ce7e72e73e13881"
server_key_version: "encryptionkey@0"
I20260811 22:30:07.866571 16397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:07.871919 16397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:07.875224 16397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:07.875458 16397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:07.883038 16397 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35893
I20260811 22:30:07.883114 16474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35893 every 8 connection(s)
I20260811 22:30:07.884205 16397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:07.887342 16476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:07.889264 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 16397
I20260811 22:30:07.889420 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:07.889763 24863 external_mini_cluster.cc:1428] Setting key d3324d7e08648825290eddecab1f85c0
I20260811 22:30:07.894506 16476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca: Bootstrap starting.
I20260811 22:30:07.895309 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:07.891264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:55747 (local address 127.0.0.1:35893)
0811 22:30:07.891692 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:07.891708 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:07.891744 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:30:07.892457 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:07.892475 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:07.892767 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:07.893051 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:07.893958 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:07.893963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:07.893976 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:07.893986 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:07.894044 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:07.894064 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:07.894177 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:07.894304 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 22:30:07.894481 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":129,"threads_started":1}
I20260811 22:30:07.896958 16476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:07.897837 16476 log.cc:824] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:07.898270 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:07.900520 16476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca: No bootstrap required, opened a new log
I20260811 22:30:07.902292 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:07.900632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:41679 (local address 127.0.0.1:35893)
0811 22:30:07.900747 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:07.900752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:07.900939 (+ 187us) server_negotiation.cc:407] Connection header received
0811 22:30:07.901021 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:07.901024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:07.901060 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:07.901149 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:07.901906 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:07.901910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:07.901912 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:07.901925 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:07.901954 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:07.901969 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:07.902061 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:07.902122 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 22:30:07.902163 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 22:30:07.904881 16476 raft_consensus.cc:348] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b521460ae73648a8802a0a2eff324aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35893 } }
I20260811 22:30:07.905131 16476 raft_consensus.cc:374] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:07.905167 16476 raft_consensus.cc:729] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b521460ae73648a8802a0a2eff324aca, State: Initialized, Role: FOLLOWER
I20260811 22:30:07.905557 16476 consensus_queue.cc:260] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b521460ae73648a8802a0a2eff324aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35893 } }
I20260811 22:30:07.905673 16476 raft_consensus.cc:388] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:07.905714 16476 raft_consensus.cc:482] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:07.905786 16476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:07.906744 16476 raft_consensus.cc:504] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b521460ae73648a8802a0a2eff324aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35893 } }
I20260811 22:30:07.907107 16476 leader_election.cc:302] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b521460ae73648a8802a0a2eff324aca; no voters:
I20260811 22:30:07.907401 16476 leader_election.cc:288] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:07.907619 16488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:07.907999 16488 raft_consensus.cc:686] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [term 1 LEADER]: Becoming Leader. State: Replica: b521460ae73648a8802a0a2eff324aca, State: Running, Role: LEADER
I20260811 22:30:07.908602 16488 consensus_queue.cc:237] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b521460ae73648a8802a0a2eff324aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35893 } }
I20260811 22:30:07.908882 16476 sys_catalog.cc:564] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:07.910249 16489 sys_catalog.cc:455] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b521460ae73648a8802a0a2eff324aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b521460ae73648a8802a0a2eff324aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35893 } } }
I20260811 22:30:07.910388 16489 sys_catalog.cc:458] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:07.910801 16499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:07.910750 16490 sys_catalog.cc:455] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b521460ae73648a8802a0a2eff324aca. Latest consensus state: current_term: 1 leader_uuid: "b521460ae73648a8802a0a2eff324aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b521460ae73648a8802a0a2eff324aca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35893 } } }
I20260811 22:30:07.910923 16490 sys_catalog.cc:458] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:07.913713 16499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:07.919739 16499 catalog_manager.cc:1347] Generated new cluster ID: d05bda2cc7794cdf81bb2c1c98a4ed1d
I20260811 22:30:07.920149 16499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:07.930644 16499 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:07.931039 16499 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:07.939385 16499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b521460ae73648a8802a0a2eff324aca: Generated new TSK 0
I20260811 22:30:07.940266 16499 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:07.948019 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 16397
2026-08-11T22:30:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:08 dist-test-slave-mvvj krb5kdc[16512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:08 dist-test-slave-mvvj krb5kdc[16512](info): set up 2 sockets
Aug 11 22:30:08 dist-test-slave-mvvj krb5kdc[16512](info): commencing operation
krb5kdc: starting...
W20260811 22:30:09.110932 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:09 dist-test-slave-mvvj krb5kdc[16512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:30:09.256400 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42295
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:42295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:09.375283 16526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:09.375659 16526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:09.375754 16526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:09.380142 16526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:09.380271 16526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:09.380311 16526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:09.380350 16526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:09.380385 16526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:09.386397 16526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:42295
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:42295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:09.387655 16526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:09.388821 16526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:30:09.399168 16526 server_base.cc:1034] running on GCE node
W20260811 22:30:09.399051 16534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:09.399087 16531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:09.399319 16532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:09.400084 16526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:09.401417 16526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:09.402660 16526 hybrid_clock.cc:648] HybridClock initialized: now 1786487409402623 us; error 49 us; skew 500 ppm
Aug 11 22:30:09 dist-test-slave-mvvj krb5kdc[16512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:09.405990 16526 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:30:09.407137 16526 webserver.cc:460] Webserver started at http://127.24.71.254:34087/ using document root <none> and password file <none>
I20260811 22:30:09.407799 16526 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:09.407943 16526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:09.408234 16526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:09.410672 16526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "dabb2c1cf11f4d78a8eaea143fb605fb"
format_stamp: "Formatted at 2026-08-11 22:30:09 on dist-test-slave-mvvj"
server_key: "6d33b28b222c07a8a935083033024d10"
server_key_iv: "e78054603046d5e337322eb7408cf58f"
server_key_version: "encryptionkey@0"
I20260811 22:30:09.411369 16526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "dabb2c1cf11f4d78a8eaea143fb605fb"
format_stamp: "Formatted at 2026-08-11 22:30:09 on dist-test-slave-mvvj"
server_key: "6d33b28b222c07a8a935083033024d10"
server_key_iv: "e78054603046d5e337322eb7408cf58f"
server_key_version: "encryptionkey@0"
I20260811 22:30:09.416172 16526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:30:09.419219 16543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:09.420912 16526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:09.421109 16526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "dabb2c1cf11f4d78a8eaea143fb605fb"
format_stamp: "Formatted at 2026-08-11 22:30:09 on dist-test-slave-mvvj"
server_key: "6d33b28b222c07a8a935083033024d10"
server_key_iv: "e78054603046d5e337322eb7408cf58f"
server_key_version: "encryptionkey@0"
I20260811 22:30:09.421260 16526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:09.444061 16526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:09.448182 16526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:09.448463 16526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:09.457803 16526 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:42295
I20260811 22:30:09.457957 16605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:42295 every 8 connection(s)
I20260811 22:30:09.459426 16526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:09.463781 16606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:09.465281 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 16526
I20260811 22:30:09.465430 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:09.465710 24863 external_mini_cluster.cc:1428] Setting key 471998a108062d82831f221a1928673a
I20260811 22:30:09.471290 16606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb: Bootstrap starting.
I20260811 22:30:09.474290 16606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:09.475260 16606 log.cc:824] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:30:09 dist-test-slave-mvvj krb5kdc[16512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:30:09.477931 16606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb: No bootstrap required, opened a new log
I20260811 22:30:09.483114 16606 raft_consensus.cc:348] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabb2c1cf11f4d78a8eaea143fb605fb" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42295 } }
I20260811 22:30:09.483420 16606 raft_consensus.cc:374] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:09.483471 16606 raft_consensus.cc:729] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dabb2c1cf11f4d78a8eaea143fb605fb, State: Initialized, Role: FOLLOWER
I20260811 22:30:09.484026 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:09.467339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36745 (local address 127.24.71.254:42295)
0811 22:30:09.467887 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:09.467909 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:09.468129 (+ 220us) server_negotiation.cc:407] Connection header received
0811 22:30:09.469100 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:09.469127 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:09.469649 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:09.470061 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:09.470828 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:09.472407 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:09.473412 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:09.474115 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:09.474292 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:09.477693 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:09.477700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:09.477718 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:09.477750 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:09.480124 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:09.480792 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:09.480799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:09.480805 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:09.480962 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:09.481429 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:09.481433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:09.481435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:09.482015 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:09.482265 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:09.482636 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 22:30:09.482888 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":138,"threads_started":1}
I20260811 22:30:09.484026 16606 consensus_queue.cc:260] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabb2c1cf11f4d78a8eaea143fb605fb" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42295 } }
I20260811 22:30:09.484218 16606 raft_consensus.cc:388] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:09.484280 16606 raft_consensus.cc:482] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:09.484378 16606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:09.485693 16606 raft_consensus.cc:504] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabb2c1cf11f4d78a8eaea143fb605fb" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42295 } }
I20260811 22:30:09.486066 16606 leader_election.cc:302] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dabb2c1cf11f4d78a8eaea143fb605fb; no voters:
I20260811 22:30:09.486430 16606 leader_election.cc:288] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:09.486881 16611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:09.487138 16611 raft_consensus.cc:686] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [term 1 LEADER]: Becoming Leader. State: Replica: dabb2c1cf11f4d78a8eaea143fb605fb, State: Running, Role: LEADER
I20260811 22:30:09.487489 16611 consensus_queue.cc:237] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabb2c1cf11f4d78a8eaea143fb605fb" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42295 } }
I20260811 22:30:09.487792 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:09.487962 16606 sys_catalog.cc:564] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:09.489400 16613 sys_catalog.cc:455] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader dabb2c1cf11f4d78a8eaea143fb605fb. Latest consensus state: current_term: 1 leader_uuid: "dabb2c1cf11f4d78a8eaea143fb605fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabb2c1cf11f4d78a8eaea143fb605fb" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42295 } } }
I20260811 22:30:09.489542 16613 sys_catalog.cc:458] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:09.489874 16612 sys_catalog.cc:455] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dabb2c1cf11f4d78a8eaea143fb605fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dabb2c1cf11f4d78a8eaea143fb605fb" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 42295 } } }
I20260811 22:30:09.489974 16612 sys_catalog.cc:458] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb [sys.catalog]: This master's current role is: LEADER
W20260811 22:30:09.493515 16631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:09.493608 16631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:09.493674 16626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:09.494643 16626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:09.500453 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:09.490265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37773 (local address 127.24.71.254:42295)
0811 22:30:09.490428 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:09.490432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:09.491023 (+ 591us) server_negotiation.cc:407] Connection header received
0811 22:30:09.491211 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:09.491215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:09.491265 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:09.491351 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:09.492373 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:09.493122 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:09.494275 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:09.494622 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:09.494706 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:09.496754 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:09.496758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:09.496760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:09.496794 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:09.498380 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:09.498940 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:09.498944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:09.498946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:09.499038 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:09.499390 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:09.499393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:09.499395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:09.499586 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:09.499741 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:09.500150 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 22:30:09.500270 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:30:09.505687 16626 catalog_manager.cc:1347] Generated new cluster ID: 4b6928e39b8746889928ca270c4976f4
I20260811 22:30:09.505786 16626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:09.548601 16626 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:09.549613 16626 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:09.555110 16626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dabb2c1cf11f4d78a8eaea143fb605fb: Generated new TSK 0
I20260811 22:30:09.556150 16626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:09.560801 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 16526
W20260811 22:30:09.569977 16510 connection.cc:535] client connection to 127.24.71.254:42295 recv error: Network error: failed to read from TLS socket (remote: 127.24.71.254:42295): Connection reset by peer (error 104)
2026-08-11T22:30:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1613 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:09 dist-test-slave-mvvj krb5kdc[16638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:09 dist-test-slave-mvvj krb5kdc[16638](info): set up 2 sockets
Aug 11 22:30:09 dist-test-slave-mvvj krb5kdc[16638](info): commencing operation
krb5kdc: starting...
W20260811 22:30:10.679593 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:10 dist-test-slave-mvvj krb5kdc[16638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:30:10.820572 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:37673
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:37673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:10.940932 16653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:10.941238 16653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:10.941282 16653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:10.945387 16653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:10.945466 16653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:10.945487 16653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:10.945506 16653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:10.945523 16653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:10.951227 16653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:37673
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:37673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:10.952453 16653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:10.953548 16653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:10.963038 16664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:10.963091 16653 server_base.cc:1034] running on GCE node
W20260811 22:30:10.963039 16659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:10.963044 16660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:10.964143 16653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:10.965457 16653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:10.966661 16653 hybrid_clock.cc:648] HybridClock initialized: now 1786487410966640 us; error 40 us; skew 500 ppm
Aug 11 22:30:10 dist-test-slave-mvvj krb5kdc[16638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:10.974771 16653 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:30:10.976032 16653 webserver.cc:460] Webserver started at http://127.24.71.254:41105/ using document root <none> and password file <none>
I20260811 22:30:10.976655 16653 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:10.976719 16653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:10.976907 16653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:10.979245 16653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "fd725263100147549ac9e329d8272758"
format_stamp: "Formatted at 2026-08-11 22:30:10 on dist-test-slave-mvvj"
server_key: "b991bd11c22e1f3fb1ed72750ed80eba"
server_key_iv: "bb469566b876d41e63f048d71ad02202"
server_key_version: "encryptionkey@0"
I20260811 22:30:10.979966 16653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "fd725263100147549ac9e329d8272758"
format_stamp: "Formatted at 2026-08-11 22:30:10 on dist-test-slave-mvvj"
server_key: "b991bd11c22e1f3fb1ed72750ed80eba"
server_key_iv: "bb469566b876d41e63f048d71ad02202"
server_key_version: "encryptionkey@0"
I20260811 22:30:10.984993 16653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:30:10.987993 16673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:10.989574 16653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:30:10.989771 16653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "fd725263100147549ac9e329d8272758"
format_stamp: "Formatted at 2026-08-11 22:30:10 on dist-test-slave-mvvj"
server_key: "b991bd11c22e1f3fb1ed72750ed80eba"
server_key_iv: "bb469566b876d41e63f048d71ad02202"
server_key_version: "encryptionkey@0"
I20260811 22:30:10.989923 16653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:11.005550 16653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:11.009565 16653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:11.009832 16653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:11.018164 16653 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:37673
I20260811 22:30:11.018182 16734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:37673 every 8 connection(s)
I20260811 22:30:11.019301 16653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:11.019773 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 16653
I20260811 22:30:11.019977 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:11.020299 24863 external_mini_cluster.cc:1428] Setting key 93bb973be80435159bc7585f24f22490
I20260811 22:30:11.022924 16735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:11.029412 16735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758: Bootstrap starting.
I20260811 22:30:11.030758 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:11.021816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38401 (local address 127.24.71.254:37673)
0811 22:30:11.022280 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:11.022316 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:11.022575 (+ 259us) server_negotiation.cc:407] Connection header received
0811 22:30:11.023501 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:11.023521 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:11.023963 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:11.024341 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:11.024787 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:11.025836 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:11.026865 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:11.027387 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:11.027578 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:11.028803 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:11.028809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:11.028828 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:11.029280 (+ 452us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:11.029409 (+ 129us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:11.029437 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:11.029596 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:11.029801 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 22:30:11.029967 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":159,"threads_started":1}
I20260811 22:30:11.032255 16735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:11.033186 16735 log.cc:824] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:11.034353 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:11.035699 16735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758: No bootstrap required, opened a new log
I20260811 22:30:11.040663 16735 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd725263100147549ac9e329d8272758" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37673 } }
I20260811 22:30:11.041062 16735 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:11.041153 16735 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd725263100147549ac9e329d8272758, State: Initialized, Role: FOLLOWER
I20260811 22:30:11.041316 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:11.036596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52363 (local address 127.24.71.254:37673)
0811 22:30:11.036732 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:11.036737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:11.036932 (+ 195us) server_negotiation.cc:407] Connection header received
0811 22:30:11.037039 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:11.037042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:11.037091 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:11.037169 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:11.037431 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:11.038316 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:11.039194 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:11.039543 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:11.039663 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:11.040664 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:11.040668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:11.040671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:11.040689 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:11.040724 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:11.040738 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:11.040839 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:11.041008 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 22:30:11.041052 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:30:11.041638 16735 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd725263100147549ac9e329d8272758" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37673 } }
I20260811 22:30:11.041816 16735 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:11.041913 16735 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:11.042023 16735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:11.043069 16735 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd725263100147549ac9e329d8272758" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37673 } }
I20260811 22:30:11.043454 16735 leader_election.cc:302] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd725263100147549ac9e329d8272758; no voters:
I20260811 22:30:11.043816 16735 leader_election.cc:288] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:11.044037 16747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:11.044276 16747 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [term 1 LEADER]: Becoming Leader. State: Replica: fd725263100147549ac9e329d8272758, State: Running, Role: LEADER
I20260811 22:30:11.044620 16747 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd725263100147549ac9e329d8272758" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37673 } }
I20260811 22:30:11.045317 16735 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:11.046437 16749 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd725263100147549ac9e329d8272758. Latest consensus state: current_term: 1 leader_uuid: "fd725263100147549ac9e329d8272758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd725263100147549ac9e329d8272758" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37673 } } }
I20260811 22:30:11.046572 16749 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:11.046963 16748 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd725263100147549ac9e329d8272758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd725263100147549ac9e329d8272758" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37673 } } }
I20260811 22:30:11.047142 16748 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:11.048110 16759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:11.051357 16759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:11.057974 16759 catalog_manager.cc:1347] Generated new cluster ID: 61b3a8e3e2ea427e865e1ff8dce0aa5b
I20260811 22:30:11.058068 16759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:11.066249 16759 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:11.067636 16759 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:11.080363 16759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd725263100147549ac9e329d8272758: Generated new TSK 0
I20260811 22:30:11.081209 16759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:11.093670 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 16653
W20260811 22:30:11.102275 16636 connection.cc:535] client connection to 127.24.71.254:37673 recv error: Network error: failed to read from TLS socket (remote: 127.24.71.254:37673): Connection reset by peer (error 104)
2026-08-11T22:30:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:11 dist-test-slave-mvvj krb5kdc[16775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:11 dist-test-slave-mvvj krb5kdc[16775](info): set up 2 sockets
Aug 11 22:30:11 dist-test-slave-mvvj krb5kdc[16775](info): commencing operation
krb5kdc: starting...
W20260811 22:30:12.244741 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:12 dist-test-slave-mvvj krb5kdc[16775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:30:12.387584 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:45143
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:45143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:12.511055 16788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:12.511365 16788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:12.511409 16788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:12.516378 16788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:12.516458 16788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:12.516479 16788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:12.516496 16788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:12.516512 16788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:12.522352 16788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:45143
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:45143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:12.523623 16788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:12.524781 16788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:12.533782 16796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:12.533813 16794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:12.533782 16798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:12.534173 16788 server_base.cc:1034] running on GCE node
I20260811 22:30:12.534873 16788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:12.536317 16788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:12.537528 16788 hybrid_clock.cc:648] HybridClock initialized: now 1786487412537506 us; error 34 us; skew 500 ppm
Aug 11 22:30:12 dist-test-slave-mvvj krb5kdc[16775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487412, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:12.540727 16788 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:30:12.541826 16788 webserver.cc:460] Webserver started at http://127.24.71.254:38385/ using document root <none> and password file <none>
I20260811 22:30:12.542433 16788 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:12.542503 16788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:12.542696 16788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:12.545115 16788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "50e5330e33e742a1a8f7511256b7945e"
format_stamp: "Formatted at 2026-08-11 22:30:12 on dist-test-slave-mvvj"
server_key: "69d0bcae84b33264af422c480183fbd0"
server_key_iv: "d5fc3a489e96cca859cffcbdb54bcc16"
server_key_version: "encryptionkey@0"
I20260811 22:30:12.545794 16788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "50e5330e33e742a1a8f7511256b7945e"
format_stamp: "Formatted at 2026-08-11 22:30:12 on dist-test-slave-mvvj"
server_key: "69d0bcae84b33264af422c480183fbd0"
server_key_iv: "d5fc3a489e96cca859cffcbdb54bcc16"
server_key_version: "encryptionkey@0"
I20260811 22:30:12.550194 16788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:12.553045 16806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:12.554592 16788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:12.554770 16788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "50e5330e33e742a1a8f7511256b7945e"
format_stamp: "Formatted at 2026-08-11 22:30:12 on dist-test-slave-mvvj"
server_key: "69d0bcae84b33264af422c480183fbd0"
server_key_iv: "d5fc3a489e96cca859cffcbdb54bcc16"
server_key_version: "encryptionkey@0"
I20260811 22:30:12.554909 16788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:12.560000 16788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:12.563325 16788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:12.563546 16788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:12.572508 16788 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:45143
I20260811 22:30:12.572539 16862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:45143 every 8 connection(s)
I20260811 22:30:12.573573 16788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:12.576166 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 16788
I20260811 22:30:12.576372 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:12.576689 24863 external_mini_cluster.cc:1428] Setting key 43fa9684ae99184e856806622ba9d1fa
I20260811 22:30:12.576756 16863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:12.583506 16863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e: Bootstrap starting.
I20260811 22:30:12.583920 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:12.578142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39361 (local address 127.24.71.254:45143)
0811 22:30:12.578664 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:12.578683 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:12.580249 (+ 1566us) server_negotiation.cc:407] Connection header received
0811 22:30:12.581021 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:12.581038 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:12.581373 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:12.581660 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:12.582538 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:12.582543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:12.582555 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:12.582566 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:12.582623 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:12.582641 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:12.582760 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:12.582873 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 22:30:12.583060 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":153,"threads_started":1}
I20260811 22:30:12.586221 16863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:12.587110 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:12.587137 16863 log.cc:824] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:12.589370 16863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e: No bootstrap required, opened a new log
I20260811 22:30:12.591287 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:12.589447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49805 (local address 127.24.71.254:45143)
0811 22:30:12.589605 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:12.589610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:12.589784 (+ 174us) server_negotiation.cc:407] Connection header received
0811 22:30:12.589919 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:12.589921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:12.589952 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:12.590020 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:12.590817 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:12.590821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:12.590824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:12.590836 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:12.590876 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:12.590887 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:12.590968 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:12.591075 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 22:30:12.591108 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:30:12.594022 16863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e5330e33e742a1a8f7511256b7945e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45143 } }
I20260811 22:30:12.594319 16863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:12.594383 16863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e5330e33e742a1a8f7511256b7945e, State: Initialized, Role: FOLLOWER
I20260811 22:30:12.594825 16863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e5330e33e742a1a8f7511256b7945e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45143 } }
I20260811 22:30:12.594981 16863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:12.595095 16863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:12.595228 16863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:12.596446 16863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e5330e33e742a1a8f7511256b7945e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45143 } }
I20260811 22:30:12.596819 16863 leader_election.cc:302] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50e5330e33e742a1a8f7511256b7945e; no voters:
I20260811 22:30:12.597174 16863 leader_election.cc:288] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:12.597338 16874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:12.597604 16874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [term 1 LEADER]: Becoming Leader. State: Replica: 50e5330e33e742a1a8f7511256b7945e, State: Running, Role: LEADER
I20260811 22:30:12.598069 16874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e5330e33e742a1a8f7511256b7945e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45143 } }
I20260811 22:30:12.598600 16863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:12.600126 16876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50e5330e33e742a1a8f7511256b7945e. Latest consensus state: current_term: 1 leader_uuid: "50e5330e33e742a1a8f7511256b7945e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e5330e33e742a1a8f7511256b7945e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45143 } } }
I20260811 22:30:12.600176 16875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50e5330e33e742a1a8f7511256b7945e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e5330e33e742a1a8f7511256b7945e" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 45143 } } }
I20260811 22:30:12.600340 16876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:12.600353 16875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e [sys.catalog]: This master's current role is: LEADER
W20260811 22:30:12.604321 16891 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:12.604427 16891 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:12.604527 16889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:12.605353 16889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:12.611450 16889 catalog_manager.cc:1347] Generated new cluster ID: 206260e3d8bf4491a5ccac21f16416ed
I20260811 22:30:12.611548 16889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:12.626566 16889 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:12.626987 16889 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:12.632298 16889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50e5330e33e742a1a8f7511256b7945e: Generated new TSK 0
I20260811 22:30:12.633698 16889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:12.644016 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 16788
2026-08-11T22:30:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:12 dist-test-slave-mvvj krb5kdc[16898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:12 dist-test-slave-mvvj krb5kdc[16898](info): set up 2 sockets
Aug 11 22:30:12 dist-test-slave-mvvj krb5kdc[16898](info): commencing operation
krb5kdc: starting...
W20260811 22:30:13.787592 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:13 dist-test-slave-mvvj krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:30:13.932341 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:46313
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:42495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:46313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:14.054019 16915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:14.054384 16915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:14.054476 16915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:14.058867 16915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:14.058984 16915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:14.059026 16915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:14.059087 16915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:14.059186 16915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:14.065941 16915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:42495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:46313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:46313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:14.067266 16915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:14.068439 16915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:14.077855 16924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:14.077852 16920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:14.077850 16921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:14.078351 16915 server_base.cc:1034] running on GCE node
I20260811 22:30:14.079041 16915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:14.080391 16915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:14.081566 16915 hybrid_clock.cc:648] HybridClock initialized: now 1786487414081532 us; error 50 us; skew 500 ppm
Aug 11 22:30:14 dist-test-slave-mvvj krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:14.084647 16915 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:30:14.085629 16915 webserver.cc:460] Webserver started at http://127.24.71.254:35053/ using document root <none> and password file <none>
I20260811 22:30:14.086222 16915 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:14.086314 16915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:14.086565 16915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:14.088845 16915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "170ca1ab883b46dea4c3aafd9a625d73"
format_stamp: "Formatted at 2026-08-11 22:30:14 on dist-test-slave-mvvj"
server_key: "064634cba239e1ab65ce616875b2a23f"
server_key_iv: "a957354fff1515fc5df6865e6d39bea3"
server_key_version: "encryptionkey@0"
I20260811 22:30:14.089434 16915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "170ca1ab883b46dea4c3aafd9a625d73"
format_stamp: "Formatted at 2026-08-11 22:30:14 on dist-test-slave-mvvj"
server_key: "064634cba239e1ab65ce616875b2a23f"
server_key_iv: "a957354fff1515fc5df6865e6d39bea3"
server_key_version: "encryptionkey@0"
I20260811 22:30:14.093376 16915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:14.096151 16931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:14.097576 16915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:14.097738 16915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "170ca1ab883b46dea4c3aafd9a625d73"
format_stamp: "Formatted at 2026-08-11 22:30:14 on dist-test-slave-mvvj"
server_key: "064634cba239e1ab65ce616875b2a23f"
server_key_iv: "a957354fff1515fc5df6865e6d39bea3"
server_key_version: "encryptionkey@0"
I20260811 22:30:14.097865 16915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:14.110658 16915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:14.113927 16915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:14.114154 16915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:14.122068 16915 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:46313
I20260811 22:30:14.122093 16992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:46313 every 8 connection(s)
I20260811 22:30:14.123113 16915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:14.126046 16993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:14.130879 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 16915
I20260811 22:30:14.131043 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:14.131377 24863 external_mini_cluster.cc:1428] Setting key 2c6c1ee18813cb814fe44b425f988815
I20260811 22:30:14.132618 16993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73: Bootstrap starting.
I20260811 22:30:14.135267 16993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:14.136235 16993 log.cc:824] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:14.138394 16993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73: No bootstrap required, opened a new log
Aug 11 22:30:14 dist-test-slave-mvvj krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:30:14.143049 16993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ca1ab883b46dea4c3aafd9a625d73" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46313 } }
I20260811 22:30:14.143318 16993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:14.143411 16993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170ca1ab883b46dea4c3aafd9a625d73, State: Initialized, Role: FOLLOWER
I20260811 22:30:14.144003 16993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ca1ab883b46dea4c3aafd9a625d73" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46313 } }
I20260811 22:30:14.144182 16993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:14.144269 16993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:14.144382 16993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:14.145684 16993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ca1ab883b46dea4c3aafd9a625d73" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46313 } }
I20260811 22:30:14.146103 16993 leader_election.cc:302] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 170ca1ab883b46dea4c3aafd9a625d73; no voters:
I20260811 22:30:14.146533 16993 leader_election.cc:288] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:14.146634 16998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:14.146860 16998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [term 1 LEADER]: Becoming Leader. State: Replica: 170ca1ab883b46dea4c3aafd9a625d73, State: Running, Role: LEADER
I20260811 22:30:14.147259 16998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ca1ab883b46dea4c3aafd9a625d73" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46313 } }
I20260811 22:30:14.147581 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:14.133126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51605 (local address 127.24.71.254:46313)
0811 22:30:14.133662 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:14.133679 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:14.133720 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:30:14.134400 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:14.134417 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:14.134799 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:14.135184 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:14.136109 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:14.137164 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:14.138127 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:14.138678 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:14.138852 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:14.141971 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:14.141977 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:14.141994 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:14.142028 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:14.144019 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:14.144590 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:14.144600 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:14.144610 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:14.144760 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:14.145059 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:14.145063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:14.145064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:14.145644 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:14.145866 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:14.146175 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 22:30:14.146520 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":139,"threads_started":1}
I20260811 22:30:14.148000 16993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:14.149096 17000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 170ca1ab883b46dea4c3aafd9a625d73. Latest consensus state: current_term: 1 leader_uuid: "170ca1ab883b46dea4c3aafd9a625d73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ca1ab883b46dea4c3aafd9a625d73" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46313 } } }
I20260811 22:30:14.149235 17000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:14.149569 16999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "170ca1ab883b46dea4c3aafd9a625d73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170ca1ab883b46dea4c3aafd9a625d73" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 46313 } } }
I20260811 22:30:14.149652 16999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:14.150588 17008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:14.151230 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:14.154476 17008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:14.161140 17008 catalog_manager.cc:1347] Generated new cluster ID: a7e29caf7aae40a5ad2e9f01505d1b4f
I20260811 22:30:14.161290 17008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:14.162967 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:14.153494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56657 (local address 127.24.71.254:46313)
0811 22:30:14.153608 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:14.153613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:14.153878 (+ 265us) server_negotiation.cc:407] Connection header received
0811 22:30:14.154004 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:14.154007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:14.154057 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:14.154141 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:14.155103 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:14.155908 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:14.157146 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:14.157506 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:14.157618 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:14.159470 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:14.159473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:14.159475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:14.159497 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:14.161142 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:14.161662 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:14.161665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:14.161667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:14.161748 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:14.162077 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:14.162081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:14.162082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:14.162298 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:14.162415 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:14.162706 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 22:30:14.162818 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:30:14.175601 17008 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:14.176913 17008 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:14.184424 17008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 170ca1ab883b46dea4c3aafd9a625d73: Generated new TSK 0
I20260811 22:30:14.185252 17008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:14.191833 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 16915
2026-08-11T22:30:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:14 dist-test-slave-mvvj krb5kdc[17027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:14 dist-test-slave-mvvj krb5kdc[17027](info): set up 2 sockets
Aug 11 22:30:14 dist-test-slave-mvvj krb5kdc[17027](info): commencing operation
krb5kdc: starting...
W20260811 22:30:15.303088 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:15 dist-test-slave-mvvj krb5kdc[17027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:30:15.453063 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:34075
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:40703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:34075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:15.578575 17041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:15.579046 17041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:15.579138 17041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:15.583781 17041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:15.583911 17041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:15.583956 17041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:15.583994 17041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:15.584031 17041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:15.589912 17041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:40703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:34075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:34075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:15.591153 17041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:15.592351 17041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:15.601881 17053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:15.602007 17048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:15.601907 17049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:15.602265 17041 server_base.cc:1034] running on GCE node
I20260811 22:30:15.602984 17041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:15.604244 17041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:15.605480 17041 hybrid_clock.cc:648] HybridClock initialized: now 1786487415605466 us; error 32 us; skew 500 ppm
Aug 11 22:30:15 dist-test-slave-mvvj krb5kdc[17027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487415, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:15.608896 17041 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:30:15.610024 17041 webserver.cc:460] Webserver started at http://127.24.71.254:45221/ using document root <none> and password file <none>
I20260811 22:30:15.610653 17041 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:15.610751 17041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:15.611011 17041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:15.613610 17041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "21a751d561e34061ab1ebe1f8f6ac8bd"
format_stamp: "Formatted at 2026-08-11 22:30:15 on dist-test-slave-mvvj"
server_key: "ea0dcb86921aab19fb227f7d8ffedd0e"
server_key_iv: "7ad0e3f73542225a6845e8b9ed92b091"
server_key_version: "encryptionkey@0"
I20260811 22:30:15.614252 17041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "21a751d561e34061ab1ebe1f8f6ac8bd"
format_stamp: "Formatted at 2026-08-11 22:30:15 on dist-test-slave-mvvj"
server_key: "ea0dcb86921aab19fb227f7d8ffedd0e"
server_key_iv: "7ad0e3f73542225a6845e8b9ed92b091"
server_key_version: "encryptionkey@0"
I20260811 22:30:15.618597 17041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:15.621403 17060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:15.622730 17041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:30:15.622888 17041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "21a751d561e34061ab1ebe1f8f6ac8bd"
format_stamp: "Formatted at 2026-08-11 22:30:15 on dist-test-slave-mvvj"
server_key: "ea0dcb86921aab19fb227f7d8ffedd0e"
server_key_iv: "7ad0e3f73542225a6845e8b9ed92b091"
server_key_version: "encryptionkey@0"
I20260811 22:30:15.623023 17041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:15.632661 17041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:15.636094 17041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:15.636312 17041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:15.644043 17041 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:34075
I20260811 22:30:15.644070 17115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:34075 every 8 connection(s)
I20260811 22:30:15.645057 17041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:15.648007 17116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:15.653090 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 17041
I20260811 22:30:15.653213 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:15.653564 24863 external_mini_cluster.cc:1428] Setting key c027e1acb8308133d1085557a5d4f724
I20260811 22:30:15.654146 17116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd: Bootstrap starting.
I20260811 22:30:15.656922 17116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:15.657724 17116 log.cc:824] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:15.659775 17116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd: No bootstrap required, opened a new log
I20260811 22:30:15.662885 17119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:15.655059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50875 (local address 127.24.71.254:34075)
0811 22:30:15.655574 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:15.655592 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:15.655641 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:30:15.656411 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:15.656430 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:15.656790 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:15.657144 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:15.657576 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:15.658607 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:15.659520 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:15.660083 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:15.660259 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:15.661170 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:15.661175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:15.661188 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:15.661592 (+ 404us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:15.661650 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:15.661673 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:15.661781 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:15.661930 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 22:30:15.662154 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":148,"threads_started":1}
I20260811 22:30:15.663954 17116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a751d561e34061ab1ebe1f8f6ac8bd" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34075 } }
I20260811 22:30:15.664227 17116 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:15.664301 17116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a751d561e34061ab1ebe1f8f6ac8bd, State: Initialized, Role: FOLLOWER
I20260811 22:30:15.664717 17116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a751d561e34061ab1ebe1f8f6ac8bd" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34075 } }
I20260811 22:30:15.664865 17116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:15.664937 17116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:15.665059 17116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:15.666018 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:15.666108 17116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a751d561e34061ab1ebe1f8f6ac8bd" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34075 } }
I20260811 22:30:15.666469 17116 leader_election.cc:302] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21a751d561e34061ab1ebe1f8f6ac8bd; no voters:
I20260811 22:30:15.666851 17116 leader_election.cc:288] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:15.667032 17121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:15.667362 17121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [term 1 LEADER]: Becoming Leader. State: Replica: 21a751d561e34061ab1ebe1f8f6ac8bd, State: Running, Role: LEADER
I20260811 22:30:15.667758 17121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a751d561e34061ab1ebe1f8f6ac8bd" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34075 } }
I20260811 22:30:15.668377 17116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:15.669736 17123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a751d561e34061ab1ebe1f8f6ac8bd. Latest consensus state: current_term: 1 leader_uuid: "21a751d561e34061ab1ebe1f8f6ac8bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a751d561e34061ab1ebe1f8f6ac8bd" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34075 } } }
I20260811 22:30:15.669847 17122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21a751d561e34061ab1ebe1f8f6ac8bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a751d561e34061ab1ebe1f8f6ac8bd" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 34075 } } }
I20260811 22:30:15.669907 17123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:15.669921 17122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:15.674202 17119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:15.668478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39203 (local address 127.24.71.254:34075)
0811 22:30:15.668589 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:15.668593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:15.669034 (+ 441us) server_negotiation.cc:407] Connection header received
0811 22:30:15.669526 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:15.669531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:15.669600 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:15.669665 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:15.670581 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:15.671353 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:15.672254 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:15.672581 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:15.672691 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:15.673711 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:15.673715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:15.673717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:15.673732 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:15.673764 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:15.673778 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:15.673857 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:15.673986 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 22:30:15.674046 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 22:30:15.674290 17145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:15.674347 17145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:15.674437 17138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:15.675374 17138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:15.681423 17138 catalog_manager.cc:1347] Generated new cluster ID: 9ebfc5c1935e483da08dac36468a5d9f
I20260811 22:30:15.681516 17138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:15.695142 17138 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:15.696307 17138 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:15.700947 17138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21a751d561e34061ab1ebe1f8f6ac8bd: Generated new TSK 0
I20260811 22:30:15.701860 17138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:15.713482 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 17041
2026-08-11T22:30:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:15 dist-test-slave-mvvj krb5kdc[17151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:15 dist-test-slave-mvvj krb5kdc[17151](info): set up 2 sockets
Aug 11 22:30:15 dist-test-slave-mvvj krb5kdc[17151](info): commencing operation
krb5kdc: starting...
W20260811 22:30:16.827409 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:16 dist-test-slave-mvvj krb5kdc[17151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:30:16.976435 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:35747
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:17.095377 17164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:17.095701 17164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:17.095748 17164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:17.099988 17164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:17.100072 17164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:17.100092 17164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:17.100111 17164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:17.100126 17164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:17.105787 17164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:35747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:35747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:17.107021 17164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:17.108155 17164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:17.116987 17170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:17.117049 17172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:17.117048 17169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:17.117427 17164 server_base.cc:1034] running on GCE node
I20260811 22:30:17.118011 17164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:17.119165 17164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:17.120381 17164 hybrid_clock.cc:648] HybridClock initialized: now 1786487417120352 us; error 37 us; skew 500 ppm
Aug 11 22:30:17 dist-test-slave-mvvj krb5kdc[17151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487417, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:17.128254 17164 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:30:17.129346 17164 webserver.cc:460] Webserver started at http://127.24.71.254:44839/ using document root <none> and password file <none>
I20260811 22:30:17.129981 17164 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:17.130084 17164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:17.130326 17164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:17.132648 17164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf"
format_stamp: "Formatted at 2026-08-11 22:30:17 on dist-test-slave-mvvj"
server_key: "8d4ad539e7f62cd663326e1ecd4d00e6"
server_key_iv: "d796e1d92dc2e1ca9d409ff99b38ece7"
server_key_version: "encryptionkey@0"
I20260811 22:30:17.133263 17164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf"
format_stamp: "Formatted at 2026-08-11 22:30:17 on dist-test-slave-mvvj"
server_key: "8d4ad539e7f62cd663326e1ecd4d00e6"
server_key_iv: "d796e1d92dc2e1ca9d409ff99b38ece7"
server_key_version: "encryptionkey@0"
I20260811 22:30:17.137465 17164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:17.140482 17179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:17.141961 17164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:30:17.142149 17164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf"
format_stamp: "Formatted at 2026-08-11 22:30:17 on dist-test-slave-mvvj"
server_key: "8d4ad539e7f62cd663326e1ecd4d00e6"
server_key_iv: "d796e1d92dc2e1ca9d409ff99b38ece7"
server_key_version: "encryptionkey@0"
I20260811 22:30:17.142300 17164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:17.147742 17164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:17.151176 17164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:17.151409 17164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:17.159808 17164 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:35747
I20260811 22:30:17.159866 17240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:35747 every 8 connection(s)
I20260811 22:30:17.160964 17164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:17.164048 17241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:17.164597 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 17164
I20260811 22:30:17.164742 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:17.165064 24863 external_mini_cluster.cc:1428] Setting key a760ff13cddc06fc49184434e7672acc
I20260811 22:30:17.170792 17241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf: Bootstrap starting.
I20260811 22:30:17.171157 17244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:17.166707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48097 (local address 127.24.71.254:35747)
0811 22:30:17.167226 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:17.167243 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:17.167283 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:30:17.168169 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:17.168190 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:17.168514 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:17.168914 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:17.169804 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:17.169809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:17.169823 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:17.169836 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:17.169897 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:17.169918 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:17.170086 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:17.170167 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 22:30:17.170350 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":148,"threads_started":1}
I20260811 22:30:17.173534 17241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:17.174242 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:17.174351 17241 log.cc:824] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:17.176800 17241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf: No bootstrap required, opened a new log
I20260811 22:30:17.178395 17244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:17.176447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55125 (local address 127.24.71.254:35747)
0811 22:30:17.176630 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:17.176634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:17.176895 (+ 261us) server_negotiation.cc:407] Connection header received
0811 22:30:17.177077 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:17.177080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:17.177123 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:17.177208 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:17.177947 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:17.177951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:17.177953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:30:17.177965 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:17.177997 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:30:17.178015 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:17.178107 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:17.178205 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 22:30:17.178251 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 22:30:17.181149 17241 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35747 } }
I20260811 22:30:17.181406 17241 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:30:17.181600 17241 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffa4a7cacf3b48d9ba2529bb003ca1bf, State: Initialized, Role: FOLLOWER
I20260811 22:30:17.182168 17241 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35747 } }
I20260811 22:30:17.182313 17241 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:30:17.182410 17241 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:30:17.182533 17241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:17.183642 17241 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35747 } }
I20260811 22:30:17.184067 17241 leader_election.cc:302] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffa4a7cacf3b48d9ba2529bb003ca1bf; no voters:
I20260811 22:30:17.184454 17241 leader_election.cc:288] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:30:17.184607 17252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:17.184933 17252 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [term 1 LEADER]: Becoming Leader. State: Replica: ffa4a7cacf3b48d9ba2529bb003ca1bf, State: Running, Role: LEADER
I20260811 22:30:17.185360 17252 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35747 } }
I20260811 22:30:17.186035 17241 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:17.187637 17253 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35747 } } }
I20260811 22:30:17.187687 17254 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffa4a7cacf3b48d9ba2529bb003ca1bf. Latest consensus state: current_term: 1 leader_uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4a7cacf3b48d9ba2529bb003ca1bf" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 35747 } } }
I20260811 22:30:17.187793 17253 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:17.187803 17254 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf [sys.catalog]: This master's current role is: LEADER
W20260811 22:30:17.191011 17267 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:17.191107 17267 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:17.191187 17264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:17.192107 17264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:17.199139 17264 catalog_manager.cc:1347] Generated new cluster ID: 1d390443fe914f59a20cec2fb657912b
I20260811 22:30:17.199241 17264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:17.227388 17264 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:17.227859 17264 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:17.238952 17264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffa4a7cacf3b48d9ba2529bb003ca1bf: Generated new TSK 0
I20260811 22:30:17.239990 17264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:30:17.250737 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 17164
2026-08-11T22:30:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1536 ms)
[----------] 18 tests from AuthTokenIssuingTest (28008 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:17 dist-test-slave-mvvj krb5kdc[17275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:17 dist-test-slave-mvvj krb5kdc[17275](info): set up 2 sockets
Aug 11 22:30:17 dist-test-slave-mvvj krb5kdc[17275](info): commencing operation
krb5kdc: starting...
W20260811 22:30:18.477852 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:18 dist-test-slave-mvvj krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:30:18.626896 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:41177
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:41177,127.24.71.253:42681,127.24.71.252:41681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:18.755069 17290 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:30:18.755436 17290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:18.755520 17290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:18.755604 17290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:18.760349 17290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:18.760440 17290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:18.760463 17290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:18.760516 17290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:18.760555 17290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:18.766757 17290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:41177,127.24.71.253:42681,127.24.71.252:41681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:41177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:18.768033 17290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:18.769135 17290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:18.778690 17298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:18.778698 17300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:18.778698 17297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:18.779199 17290 server_base.cc:1034] running on GCE node
I20260811 22:30:18.779978 17290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:18.781172 17290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:18.782361 17290 hybrid_clock.cc:648] HybridClock initialized: now 1786487418782330 us; error 45 us; skew 500 ppm
Aug 11 22:30:18 dist-test-slave-mvvj krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:18.785535 17290 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:30:18.786466 17290 webserver.cc:460] Webserver started at http://127.24.71.254:37193/ using document root <none> and password file <none>
I20260811 22:30:18.787020 17290 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:18.787081 17290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:18.787254 17290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:18.789419 17290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "50ff64d302f147ccb8439980075c878f"
format_stamp: "Formatted at 2026-08-11 22:30:18 on dist-test-slave-mvvj"
server_key: "ab4301dc27509422db8c154a7f274abb"
server_key_iv: "534d3492f8ed0c112806d535144fc514"
server_key_version: "encryptionkey@0"
I20260811 22:30:18.789978 17290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "50ff64d302f147ccb8439980075c878f"
format_stamp: "Formatted at 2026-08-11 22:30:18 on dist-test-slave-mvvj"
server_key: "ab4301dc27509422db8c154a7f274abb"
server_key_iv: "534d3492f8ed0c112806d535144fc514"
server_key_version: "encryptionkey@0"
I20260811 22:30:18.794276 17290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:30:18.797292 17308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:18.798897 17290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:30:18.799037 17290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "50ff64d302f147ccb8439980075c878f"
format_stamp: "Formatted at 2026-08-11 22:30:18 on dist-test-slave-mvvj"
server_key: "ab4301dc27509422db8c154a7f274abb"
server_key_iv: "534d3492f8ed0c112806d535144fc514"
server_key_version: "encryptionkey@0"
I20260811 22:30:18.799156 17290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:18.850728 17290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:18.855141 17290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:18.855345 17290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:18.864100 17290 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:41177
I20260811 22:30:18.864421 17365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:41177 every 8 connection(s)
I20260811 22:30:18.865265 17290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:18.866533 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 17290
I20260811 22:30:18.866662 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:18.866954 24863 external_mini_cluster.cc:1428] Setting key 81692bf60d7abe08f1a63f60550d6091
I20260811 22:30:18.868605 17366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:18.874691 17366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.71.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.253@KRBTEST.COM" created.
Aug 11 22:30:18 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:30:18.895758 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:18.878157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:41177 (local address 127.0.0.1:38654)
0811 22:30:18.879106 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:18.879171 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:18.880313 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:18.881262 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:18.881278 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:18.881951 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:18.882423 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:18.882448 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:18.883750 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:18.883753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:18.884627 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:18.884637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:18.885620 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:18.885623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:18.885733 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:18.886512 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:18.886531 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:18.888577 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:18.891628 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:18.891639 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:18.891649 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:18.892054 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:18.892539 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:18.892542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:18.892544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:18.892693 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:18.893545 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:18.893552 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:18.893700 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 22:30:18.893919 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 22:30:18.894479 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":80,"spinlock_wait_cycles":61312,"thread_start_us":197,"threads_started":1}
I20260811 22:30:18.895762 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:18.878157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.24.71.254:41177)
0811 22:30:18.879126 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:18.879148 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:18.879296 (+ 148us) server_negotiation.cc:407] Connection header received
0811 22:30:18.880674 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:18.880701 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:18.881008 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:18.881222 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:18.882678 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:18.883597 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:18.884755 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:18.885333 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:18.885526 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:18.888800 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:18.888813 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:18.888837 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:18.888889 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:18.891452 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:18.892225 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:18.892235 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:18.892237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:18.892411 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:18.892821 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:18.892826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:18.892829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:18.893387 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:18.893521 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:18.894009 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 22:30:18.894531 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":216,"server-negotiator.queue_time_us":564,"thread_start_us":178,"threads_started":1}
I20260811 22:30:18.898921 17366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:18.899976 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:18.899568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42681 (local address 127.0.0.1:59678)
0811 22:30:18.899723 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:18.899815 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 22:30:18.900197 17312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
I20260811 22:30:18.901232 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:18.900980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42681 (local address 127.0.0.1:59690)
0811 22:30:18.901108 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:18.901168 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:30:18.901422 17366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:42681: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab.
Entry for principal kudu/127.24.71.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab.
I20260811 22:30:18.903780 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.253:42681
--webserver_interface=127.24.71.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab
--principal=kudu/127.24.71.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:41177,127.24.71.253:42681,127.24.71.252:41681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:30:18.950678 17366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } attempt: 1
I20260811 22:30:18.951706 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:18.951339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42681 (local address 127.0.0.1:59696)
0811 22:30:18.951561 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:18.951626 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 22:30:18.952462 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:18.952238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42681 (local address 127.0.0.1:59706)
0811 22:30:18.952347 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:18.952399 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:30:18.952589 17366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:42681: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
I20260811 22:30:19.020748 17366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } attempt: 2
I20260811 22:30:19.021795 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.021389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42681 (local address 127.0.0.1:59722)
0811 22:30:19.021637 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.021702 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260811 22:30:19.022703 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.022449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42681 (local address 127.0.0.1:59732)
0811 22:30:19.022575 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.022635 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:30:19.022873 17366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:42681: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42681: connect: Connection refused (error 111)
W20260811 22:30:19.026826 17374 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:30:19.027148 17374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:19.027204 17374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:19.027293 17374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:19.031556 17374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:19.031629 17374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:19.031649 17374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:19.031666 17374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:19.031682 17374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:19.037612 17374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:41177,127.24.71.253:42681,127.24.71.252:41681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab
--principal=kudu/127.24.71.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.253:42681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.71.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:19.038866 17374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:19.040023 17374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:30:19.049381 17374 server_base.cc:1034] running on GCE node
W20260811 22:30:19.049312 17384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:19.049319 17380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:19.049315 17381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:19.050274 17374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:19.051618 17374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:19.052834 17374 hybrid_clock.cc:648] HybridClock initialized: now 1786487419052813 us; error 71 us; skew 500 ppm
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487419, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:19.056013 17374 init.cc:377] Logged in from keytab as kudu/127.24.71.253@KRBTEST.COM (short username kudu)
I20260811 22:30:19.056996 17374 webserver.cc:460] Webserver started at http://127.24.71.253:38031/ using document root <none> and password file <none>
I20260811 22:30:19.057601 17374 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:19.057695 17374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:19.057948 17374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:19.060338 17374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/data/instance:
uuid: "903dee9da88d491a8c774e827bc2883d"
format_stamp: "Formatted at 2026-08-11 22:30:19 on dist-test-slave-mvvj"
server_key: "727b87af3da2d4c6203e0ff2322f9670"
server_key_iv: "f2a0d04997e67f391ffacd557645c019"
server_key_version: "encryptionkey@0"
I20260811 22:30:19.060984 17374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/wal/instance:
uuid: "903dee9da88d491a8c774e827bc2883d"
format_stamp: "Formatted at 2026-08-11 22:30:19 on dist-test-slave-mvvj"
server_key: "727b87af3da2d4c6203e0ff2322f9670"
server_key_iv: "f2a0d04997e67f391ffacd557645c019"
server_key_version: "encryptionkey@0"
I20260811 22:30:19.065193 17374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:30:19.067989 17392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:19.069368 17374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:30:19.069543 17374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/wal
uuid: "903dee9da88d491a8c774e827bc2883d"
format_stamp: "Formatted at 2026-08-11 22:30:19 on dist-test-slave-mvvj"
server_key: "727b87af3da2d4c6203e0ff2322f9670"
server_key_iv: "f2a0d04997e67f391ffacd557645c019"
server_key_version: "encryptionkey@0"
I20260811 22:30:19.069680 17374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:19.086674 17374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:19.090530 17374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:19.090767 17374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:19.098476 17374 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.253:42681
I20260811 22:30:19.098800 17453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.253:42681 every 8 connection(s)
I20260811 22:30:19.099552 17374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
I20260811 22:30:19.102041 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 17374
I20260811 22:30:19.102176 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-1/wal/instance
I20260811 22:30:19.102514 24863 external_mini_cluster.cc:1428] Setting key 5851ad851788feec0a1425d81805bc5a
I20260811 22:30:19.102893 17454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:19.107465 17454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.71.252@KRBTEST.COM; defaulting to no policy
I20260811 22:30:19.114044 17366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.252@KRBTEST.COM" created.
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487419, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
I20260811 22:30:19.125969 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.114603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42681 (local address 127.0.0.1:59738)
0811 22:30:19.114804 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.114819 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:19.114901 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:19.115509 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:19.115513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:19.115702 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:19.115798 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.115804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.116960 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.116966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.117746 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.117759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.118643 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.118647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.118746 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.119257 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:19.119272 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:19.121563 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:19.123710 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.123715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.123717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.124051 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.124477 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.124483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.124484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.124585 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.125424 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:19.125427 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:19.125538 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 22:30:19.125629 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:30:19.125798 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260811 22:30:19.126194 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.109631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.24.71.254:41177)
0811 22:30:19.109773 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.109777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.110904 (+ 1127us) server_negotiation.cc:407] Connection header received
0811 22:30:19.111948 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.111952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.112072 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.112171 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.113479 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.114117 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.116560 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.116926 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.117050 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.121108 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.121113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.121119 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.121154 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.123962 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.124562 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.124568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.124570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.124672 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.124976 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.124981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.124984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.125243 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.125378 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.125843 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.126010 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:30:19.127243 17459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.114707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.24.71.253:42681)
0811 22:30:19.115080 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.115090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.115111 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:30:19.115165 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.115170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.115350 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.115515 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.115952 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.116828 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.117887 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.118492 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.118627 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.121747 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.121755 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.121771 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.121803 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.123582 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.124174 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.124184 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.124187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.124361 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.124692 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.124696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.124697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.125278 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.126676 (+ 1398us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.126785 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.126933 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":195,"threads_started":1}
I20260811 22:30:19.127243 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.110148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:41177 (local address 127.0.0.1:38668)
0811 22:30:19.110743 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.110790 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:19.111666 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:19.112352 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:19.112360 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:19.112711 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:19.113324 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.113339 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.114296 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.114299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.116406 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.116422 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.117094 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.117098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.117335 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.118849 (+ 1514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:19.118876 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:19.120917 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:19.124084 (+ 3167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.124098 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.124112 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.124394 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.124776 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.124779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.124780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.124857 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.125366 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:19.125375 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:19.125539 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 22:30:19.125717 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 22:30:19.126097 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"spinlock_wait_cycles":4480,"thread_start_us":164,"threads_started":1}
I20260811 22:30:19.129345 17454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:19.130060 17366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:19.130908 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.130556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58086)
0811 22:30:19.130700 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.130815 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:30:19.131951 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.131591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58102)
0811 22:30:19.131742 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.131790 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 22:30:19.132164 17366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:41681: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487419, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
I20260811 22:30:19.139245 17459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.130220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59752 (local address 127.24.71.253:42681)
0811 22:30:19.130327 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.130335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.130559 (+ 224us) server_negotiation.cc:407] Connection header received
0811 22:30:19.130609 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.130612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.130662 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.130747 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.131247 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.132166 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.133041 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.133281 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.133381 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.135941 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.135946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.135949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.135978 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.137538 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.138025 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.138031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.138033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.138112 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.138389 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.138394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.138395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.138588 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.138694 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.138952 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.139117 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":28}
I20260811 22:30:19.139245 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.129997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42681 (local address 127.0.0.1:59752)
0811 22:30:19.130437 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.130451 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:19.130540 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:19.130783 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:19.130786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:19.130987 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:19.131118 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.131125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.132292 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.132295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.132898 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.132906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.133540 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.133544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.133696 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.134195 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:19.134209 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:19.135730 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:19.137679 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.137682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.137685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.137897 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.138209 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.138212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.138214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.138294 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.138681 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:19.138684 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:19.138786 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 22:30:19.138887 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:30:19.139052 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab.
Entry for principal kudu/127.24.71.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab.
I20260811 22:30:19.140460 17454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:19.141206 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.252:41681
--webserver_interface=127.24.71.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab
--principal=kudu/127.24.71.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:41177,127.24.71.253:42681,127.24.71.252:41681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:30:19.141767 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.141082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58106)
0811 22:30:19.141268 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.141645 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260811 22:30:19.142665 17395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
I20260811 22:30:19.149070 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.148708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58108)
0811 22:30:19.148898 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.148953 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260811 22:30:19.149369 17454 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:41681: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
I20260811 22:30:19.163332 17366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } attempt: 1
I20260811 22:30:19.164340 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.164026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58116)
0811 22:30:19.164185 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.164254 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:30:19.165064 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.164840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58128)
0811 22:30:19.164956 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.165009 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:30:19.165192 17366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:41681: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
I20260811 22:30:19.198709 17454 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } attempt: 1
I20260811 22:30:19.199790 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.199435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58144)
0811 22:30:19.199635 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.199703 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 22:30:19.200662 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.200377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58150)
0811 22:30:19.200545 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.200596 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260811 22:30:19.200819 17454 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:41681: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
I20260811 22:30:19.240383 17366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } attempt: 2
I20260811 22:30:19.241338 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.240997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58156)
0811 22:30:19.241164 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.241237 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 22:30:19.242098 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.241867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58162)
0811 22:30:19.241985 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.242038 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:30:19.242228 17366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:41681: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
I20260811 22:30:19.269006 17454 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } attempt: 2
I20260811 22:30:19.269980 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.269657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58164)
0811 22:30:19.269822 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.269895 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 22:30:19.270069 17461 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:30:19.270462 17461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:19.270525 17461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:19.270632 17461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:30:19.271085 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.270534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58178)
0811 22:30:19.270928 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.271006 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313}
W20260811 22:30:19.271216 17454 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:41681: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
W20260811 22:30:19.275264 17461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:19.275355 17461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:19.275406 17461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:19.275444 17461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:19.275480 17461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:19.281349 17461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:41177,127.24.71.253:42681,127.24.71.252:41681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab
--principal=kudu/127.24.71.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.252:41681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.71.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:19.282629 17461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:19.283778 17461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:19.293105 17469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:19.293112 17471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:19.293120 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:19.293440 17461 server_base.cc:1034] running on GCE node
I20260811 22:30:19.294114 17461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:19.295356 17461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:19.296553 17461 hybrid_clock.cc:648] HybridClock initialized: now 1786487419296523 us; error 43 us; skew 500 ppm
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487419, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:19.299665 17461 init.cc:377] Logged in from keytab as kudu/127.24.71.252@KRBTEST.COM (short username kudu)
I20260811 22:30:19.300745 17461 webserver.cc:460] Webserver started at http://127.24.71.252:36883/ using document root <none> and password file <none>
I20260811 22:30:19.301349 17461 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:19.301419 17461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:19.301609 17461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:19.304176 17461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/data/instance:
uuid: "4262e2126cd4413f89d8416bb1943652"
format_stamp: "Formatted at 2026-08-11 22:30:19 on dist-test-slave-mvvj"
server_key: "af5344e90156b4327206d798197993b6"
server_key_iv: "a247cb911519c2313bfad3db6ade591b"
server_key_version: "encryptionkey@0"
I20260811 22:30:19.304795 17461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/wal/instance:
uuid: "4262e2126cd4413f89d8416bb1943652"
format_stamp: "Formatted at 2026-08-11 22:30:19 on dist-test-slave-mvvj"
server_key: "af5344e90156b4327206d798197993b6"
server_key_iv: "a247cb911519c2313bfad3db6ade591b"
server_key_version: "encryptionkey@0"
I20260811 22:30:19.309106 17461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:30:19.311987 17478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:19.313463 17461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:30:19.313603 17461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/wal
uuid: "4262e2126cd4413f89d8416bb1943652"
format_stamp: "Formatted at 2026-08-11 22:30:19 on dist-test-slave-mvvj"
server_key: "af5344e90156b4327206d798197993b6"
server_key_iv: "a247cb911519c2313bfad3db6ade591b"
server_key_version: "encryptionkey@0"
I20260811 22:30:19.313716 17461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:19.341410 17366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } attempt: 3
I20260811 22:30:19.342387 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.342047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58182)
0811 22:30:19.342224 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.342296 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 22:30:19.343242 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.343042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58192)
0811 22:30:19.343141 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.343188 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:30:19.343447 17366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:41681: Network error: Client connection negotiation failed: client connection to 127.24.71.252:41681: connect: Connection refused (error 111)
I20260811 22:30:19.346588 17461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:19.350078 17461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:19.350279 17461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:19.357928 17461 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.252:41681
I20260811 22:30:19.358242 17539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.252:41681 every 8 connection(s)
I20260811 22:30:19.359016 17461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
I20260811 22:30:19.362002 17540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:19.362378 17454 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } attempt: 3
I20260811 22:30:19.365746 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 17461
I20260811 22:30:19.365896 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786487121090521-24863-0/minicluster-data/master-2/wal/instance
I20260811 22:30:19.366217 24863 external_mini_cluster.cc:1428] Setting key 85796ec32b7c9e18582cfdb23353b99c
I20260811 22:30:19.367867 17540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487419, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
I20260811 22:30:19.377270 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.363009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58200)
0811 22:30:19.363195 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.363216 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:19.363331 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:19.365822 (+ 2491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:19.365826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:19.366050 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:19.366180 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.366189 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.368206 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.368210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.369007 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.369020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.369995 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.370000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.370103 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.370677 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:19.370693 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:19.372385 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:19.375010 (+ 2625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.375014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.375016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.375302 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.375732 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.375737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.375739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.375872 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.376701 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:19.376705 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:19.376815 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 22:30:19.376942 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 22:30:19.377096 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487419, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:30:19.378609 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.363938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58200 (local address 127.24.71.252:41681)
0811 22:30:19.364428 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.364446 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.364485 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:30:19.365148 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.365165 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.365529 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.366019 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.366491 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.368036 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.369146 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.369821 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.370432 (+ 611us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.372564 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.372569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.372579 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.372606 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.374870 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.375439 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.375448 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.375456 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.375628 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.376002 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.376005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.376007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.376587 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.376721 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.377140 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.377462 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"spinlock_wait_cycles":2816,"thread_start_us":148,"threads_started":1}
I20260811 22:30:19.379084 17543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.366985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58216 (local address 127.24.71.252:41681)
0811 22:30:19.367369 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.367378 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.367587 (+ 209us) server_negotiation.cc:407] Connection header received
0811 22:30:19.367724 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.367727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.367775 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.367926 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.368800 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.369493 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.371029 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.371432 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.371546 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.374596 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.374600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.374605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.374635 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.376413 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.376999 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.377006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.377011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.377153 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.377797 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.377801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.377802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.378013 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.378202 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.378450 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.378566 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"spinlock_wait_cycles":3840,"thread_start_us":153,"threads_started":1}
I20260811 22:30:19.381971 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.370069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.24.71.254:41177)
0811 22:30:19.370202 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.370208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.370600 (+ 392us) server_negotiation.cc:407] Connection header received
0811 22:30:19.370864 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.370868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.370941 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.371030 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.372347 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.373265 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.374475 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.374743 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.374866 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.378259 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.378263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.378266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.378310 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.380011 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.380489 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.380492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.380494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.380610 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.380925 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.380928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.380930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.381181 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.381284 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.381700 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.381827 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:30:19.382519 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.370001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:41177 (local address 127.0.0.1:38672)
0811 22:30:19.370470 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.370521 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:19.370694 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:19.371130 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:19.371137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:19.371673 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:19.372188 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.372201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.373464 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.373467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.374361 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.374376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.374870 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.374874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.375024 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.375807 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:19.375862 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:19.378088 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:19.380120 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.380129 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.380136 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.380370 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.380717 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.380720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.380722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.380826 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.381305 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:19.381312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:19.381445 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 22:30:19.382119 (+ 674us) client_negotiation.cc:240] Negotiation successful
0811 22:30:19.382316 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":109,"threads_started":1}
I20260811 22:30:19.383545 17454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d: Bootstrap starting.
I20260811 22:30:19.383698 17540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:19.386304 17454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:19.387028 17454 log.cc:824] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:30:19.389425 17454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d: No bootstrap required, opened a new log
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487419, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
I20260811 22:30:19.392695 17546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.381631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.24.71.254:41177)
0811 22:30:19.381931 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.381943 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.381961 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:30:19.382025 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.382027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.382090 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.382184 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.382887 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.383689 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.384673 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.385338 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.385849 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.388469 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.388473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.388475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.388506 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.390406 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.390952 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.390956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.390957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.391055 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.391430 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.391434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.391437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.391675 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.391954 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.392340 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.392479 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":94,"threads_started":1}
I20260811 22:30:19.394165 17454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } }
I20260811 22:30:19.394440 17454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903dee9da88d491a8c774e827bc2883d, State: Initialized, Role: FOLLOWER
I20260811 22:30:19.394721 17459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.384997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.24.71.253:42681)
0811 22:30:19.385123 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.385128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.385141 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:30:19.385251 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.385255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.385307 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.385394 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.385914 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.386826 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.388094 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.388432 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.388544 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.390994 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.390998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.391001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.391030 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.392747 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.393239 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.393244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.393246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.393341 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.393695 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.393699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.393700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.393940 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.394015 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.394406 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.394533 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:30:19.394822 17454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } }
I20260811 22:30:19.395035 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.384885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42681 (local address 127.0.0.1:59766)
0811 22:30:19.385052 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.385068 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:19.385142 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:19.385432 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:19.385436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:19.385626 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:19.385773 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.385784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.386996 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.387002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.387925 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.387940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.388553 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.388559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.388673 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.389270 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:19.389290 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:19.390810 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:19.392857 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.392861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.392863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.393130 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.393476 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.393479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.393481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.393579 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.394095 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:19.394097 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:19.394201 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 22:30:19.394318 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:30:19.394635 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 22:30:19.396064 17540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:19.396072 17549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.396265 17549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:19.396485 17454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487419, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
W20260811 22:30:19.403388 17563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:19.403549 17563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:19.406013 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.393940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59780 (local address 127.24.71.253:42681)
0811 22:30:19.394444 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.394454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.394465 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:30:19.394499 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.394504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.394546 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.395436 (+ 890us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.395579 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.396336 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.397372 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.397925 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.398414 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.401071 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.401075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.401078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.401103 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.402986 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.403512 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.403516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.403519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.403658 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.404323 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.404328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.404360 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.404841 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.405621 (+ 780us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.405704 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.405833 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":139,"threads_started":1}
I20260811 22:30:19.402069 17374 master_runner.cc:186] Error getting master registration for 127.24.71.254:41177: OK, instance_id { permanent_uuid: "50ff64d302f147ccb8439980075c878f" instance_seqno: 1786487418861646 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:30:19.406196 17374 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:30:19.406931 17545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.396627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58232)
0811 22:30:19.396803 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.396839 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:19.396940 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:19.398032 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:19.398035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:19.398210 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:19.398698 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.398712 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.399902 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.399905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.400631 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.400642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.401264 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.401268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.401355 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.401845 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:19.401859 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:19.403395 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:19.405373 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.405376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.405378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.405615 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.405875 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.405877 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.405879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.405949 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.406445 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:19.406447 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:19.406533 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 22:30:19.406641 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:30:19.406780 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 22:30:19.407440 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.396658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58232 (local address 127.24.71.252:41681)
0811 22:30:19.396757 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.396762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.396900 (+ 138us) server_negotiation.cc:407] Connection header received
0811 22:30:19.397789 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.397792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.397839 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.397918 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.398837 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.399714 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.400751 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.401059 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.401156 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.403531 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.403534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.403537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.403557 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.405260 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.405709 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.405712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.405713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.405781 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.406038 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.406041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.406043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.406312 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.406411 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.406687 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.407344 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":552,"server-negotiator.queue_time_us":37}
I20260811 22:30:19.409693 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:19.410845 17540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652: Bootstrap starting.
I20260811 22:30:19.411219 17338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "50ff64d302f147ccb8439980075c878f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38678
I20260811 22:30:19.413920 17540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:19.414808 17540 log.cc:824] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:19.417261 17540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652: No bootstrap required, opened a new log
I20260811 22:30:19.421474 17540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } }
I20260811 22:30:19.421759 17540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4262e2126cd4413f89d8416bb1943652, State: Initialized, Role: FOLLOWER
I20260811 22:30:19.422117 17540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } }
I20260811 22:30:19.422648 17567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.422801 17567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:19.423553 17540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:30:19.429702 17579 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:19.429875 17579 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:19.427525 17461 master_runner.cc:186] Error getting master registration for 127.24.71.254:41177: OK, instance_id { permanent_uuid: "50ff64d302f147ccb8439980075c878f" instance_seqno: 1786487418861646 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:30:19.430972 17461 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:30:19.507650 17366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } attempt: 4
Aug 11 22:30:19 dist-test-slave-mvvj krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487418, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
I20260811 22:30:19.517616 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.508375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.24.71.252:41681)
0811 22:30:19.508514 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.508518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.508619 (+ 101us) server_negotiation.cc:407] Connection header received
0811 22:30:19.508762 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.508765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.508821 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.508916 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.509443 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.510233 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.511026 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.511374 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.511470 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.513924 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.513929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.513932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.513963 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.515727 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.516198 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.516202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.516204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.516291 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.516589 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.516594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.516596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.516826 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.516922 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.517317 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.517438 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:30:19.517632 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.508311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:41681 (local address 127.0.0.1:58246)
0811 22:30:19.508534 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:19.508552 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:19.508667 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:19.508943 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:19.508946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:19.509183 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:19.509298 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.509305 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.510354 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.510358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.510887 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:19.510896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.511619 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.511623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:19.511716 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.512320 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:19.512339 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:19.513776 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:19.515858 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.515862 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.515865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.516078 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.516398 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:19.516401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:19.516403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:19.516490 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:19.516922 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:19.516925 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:19.517030 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 22:30:19.517220 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:30:19.517447 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260811 22:30:19.521250 17366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f: Bootstrap starting.
I20260811 22:30:19.523878 17366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:19.524984 17366 log.cc:824] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:19.527520 17366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f: No bootstrap required, opened a new log
I20260811 22:30:19.530884 17366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } }
I20260811 22:30:19.531168 17366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ff64d302f147ccb8439980075c878f, State: Initialized, Role: FOLLOWER
I20260811 22:30:19.531522 17366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } }
I20260811 22:30:19.532131 17582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.532426 17582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:19.533106 17366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:19.533177 17338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:30:19.533274 17338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:19.534814 17338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } }
I20260811 22:30:19.536595 17425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50ff64d302f147ccb8439980075c878f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "903dee9da88d491a8c774e827bc2883d"
I20260811 22:30:19.536891 17425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:19.537029 17510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50ff64d302f147ccb8439980075c878f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4262e2126cd4413f89d8416bb1943652"
I20260811 22:30:19.537257 17510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:19.538332 17425 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ff64d302f147ccb8439980075c878f in term 1.
I20260811 22:30:19.538678 17510 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ff64d302f147ccb8439980075c878f in term 1.
I20260811 22:30:19.539377 17312 leader_election.cc:302] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4262e2126cd4413f89d8416bb1943652, 50ff64d302f147ccb8439980075c878f; no voters:
I20260811 22:30:19.539678 17582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:19.539942 17582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [term 1 LEADER]: Becoming Leader. State: Replica: 50ff64d302f147ccb8439980075c878f, State: Running, Role: LEADER
I20260811 22:30:19.536382 17338 leader_election.cc:288] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [CANDIDATE]: Term 1 election: Requested vote from peers 903dee9da88d491a8c774e827bc2883d (127.24.71.253:42681), 4262e2126cd4413f89d8416bb1943652 (127.24.71.252:41681)
I20260811 22:30:19.540283 17582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } }
I20260811 22:30:19.542281 17582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50ff64d302f147ccb8439980075c878f. Latest consensus state: current_term: 1 leader_uuid: "50ff64d302f147ccb8439980075c878f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.542408 17582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [sys.catalog]: This master's current role is: LEADER
W20260811 22:30:19.546144 17595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:19.546357 17595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:19.549628 17603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:19.550702 17603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:19.565438 17510 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [term 1 FOLLOWER]: Refusing update from remote peer 50ff64d302f147ccb8439980075c878f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:19.565830 17425 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [term 1 FOLLOWER]: Refusing update from remote peer 50ff64d302f147ccb8439980075c878f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:19.566589 17582 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:19.567039 17582 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [LEADER]: Connected to new peer: Peer: permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:19.571408 17567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50ff64d302f147ccb8439980075c878f. Latest consensus state: current_term: 1 leader_uuid: "50ff64d302f147ccb8439980075c878f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.571565 17567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:19.571920 17549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50ff64d302f147ccb8439980075c878f. Latest consensus state: current_term: 1 leader_uuid: "50ff64d302f147ccb8439980075c878f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.572075 17549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [sys.catalog]: This master's current role is: FOLLOWER
W20260811 22:30:19.595605 17398 tablet.cc:2369] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:30:19.597456 17610 mvcc.cc:205] Tried to move back new op lower bound from 7317452470509625344 to 7317452470440751104. Current Snapshot: MvccSnapshot[applied={T|T < 7317452470509625344}]
I20260811 22:30:19.599000 17549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50ff64d302f147ccb8439980075c878f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.599159 17549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:19.600675 17567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50ff64d302f147ccb8439980075c878f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.600826 17567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:19.601052 17596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50ff64d302f147ccb8439980075c878f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.601056 17582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50ff64d302f147ccb8439980075c878f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ff64d302f147ccb8439980075c878f" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 41177 } } peers { permanent_uuid: "903dee9da88d491a8c774e827bc2883d" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42681 } } peers { permanent_uuid: "4262e2126cd4413f89d8416bb1943652" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 41681 } } }
I20260811 22:30:19.601205 17596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:19.601233 17582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:19.601504 17605 mvcc.cc:205] Tried to move back new op lower bound from 7317452470509625344 to 7317452470440751104. Current Snapshot: MvccSnapshot[applied={T|T < 7317452470509625344}]
I20260811 22:30:19.605294 17546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.553569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.24.71.254:41177)
0811 22:30:19.553693 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.553698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.559409 (+ 5711us) server_negotiation.cc:407] Connection header received
0811 22:30:19.559917 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.559921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.559991 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.560051 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.566401 (+ 6350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.567164 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.577238 (+ 10074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.577653 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.577749 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.585521 (+ 7772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.585526 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.585529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.585564 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.587326 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.590758 (+ 3432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.590763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.590765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.590884 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.602429 (+ 11545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.602434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.602436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.602718 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.602877 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.604951 (+ 2074us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.605123 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:30:19.607168 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.555590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58250 (local address 127.24.71.252:41681)
0811 22:30:19.555711 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.555716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.561357 (+ 5641us) server_negotiation.cc:407] Connection header received
0811 22:30:19.562862 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.562866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.562924 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.562978 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.573230 (+ 10252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.574036 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.578517 (+ 4481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.578894 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.578976 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.583386 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.583391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.583394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.583435 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.585273 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.591244 (+ 5971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.591248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.591251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.591365 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.603327 (+ 11962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.603332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.603334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.603600 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.603748 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.606881 (+ 3133us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.607027 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:30:19.607686 17603 catalog_manager.cc:1347] Generated new cluster ID: d318b925fad342a0a6c1b5b1fe8db59e
I20260811 22:30:19.607752 17603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:19.609159 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:19.555000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59784 (local address 127.24.71.253:42681)
0811 22:30:19.555157 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.555162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.559650 (+ 4488us) server_negotiation.cc:407] Connection header received
0811 22:30:19.561581 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.561585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.561644 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.561697 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.567320 (+ 5623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.568115 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.575848 (+ 7733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:19.576239 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:19.576326 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:19.588396 (+ 12070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:19.588400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:19.588404 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:19.588438 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:19.590173 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.601713 (+ 11540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.601718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.601720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.601874 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:19.604082 (+ 2208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:19.604087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:19.604089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:19.604324 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:19.604439 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:19.608839 (+ 4400us) server_negotiation.cc:299] Negotiation successful
0811 22:30:19.609002 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:30:19.631901 17603 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:19.633224 17603 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:19.643879 17603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50ff64d302f147ccb8439980075c878f: Generated new TSK 0
I20260811 22:30:19.644843 17603 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:30:19.666983 17546 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.665161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38696 (local address 127.24.71.254:41177)
0811 22:30:19.665325 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.665331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.665811 (+ 480us) server_negotiation.cc:407] Connection header received
0811 22:30:19.666019 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.666023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.666074 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.666130 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.666831 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38696: BlockingRecv error: recv got EOF from 127.0.0.1:38696 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 22:30:19.667416 17542 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.665910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58252 (local address 127.24.71.252:41681)
0811 22:30:19.666044 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.666048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.666275 (+ 227us) server_negotiation.cc:407] Connection header received
0811 22:30:19.666487 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.666491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.666541 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.666599 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.667228 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58252: BlockingRecv error: recv got EOF from 127.0.0.1:58252 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:30:19.667416 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.665640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.24.71.253:42681)
0811 22:30:19.665792 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.665798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.666275 (+ 477us) server_negotiation.cc:407] Connection header received
0811 22:30:19.666488 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.666491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.666602 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.666652 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.667277 (+ 625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59792: BlockingRecv error: recv got EOF from 127.0.0.1:59792 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 22:30:19.671123 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.669896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59798 (local address 127.24.71.253:42681)
0811 22:30:19.670021 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.670025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.670131 (+ 106us) server_negotiation.cc:407] Connection header received
0811 22:30:19.670343 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.670347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.670431 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.670528 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.671007 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59798: BlockingRecv error: recv got EOF from 127.0.0.1:59798 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 22:30:19.675343 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.674411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59810 (local address 127.24.71.253:42681)
0811 22:30:19.674588 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.674591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.674604 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:30:19.674650 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.674652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.674694 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.674766 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.675227 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59810: BlockingRecv error: recv got EOF from 127.0.0.1:59810 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 22:30:19.681602 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.680243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59824 (local address 127.24.71.253:42681)
0811 22:30:19.680372 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.680375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.680653 (+ 278us) server_negotiation.cc:407] Connection header received
0811 22:30:19.680812 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.680815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.680868 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.680951 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.681427 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59824: BlockingRecv error: recv got EOF from 127.0.0.1:59824 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 22:30:19.688494 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.687401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.24.71.253:42681)
0811 22:30:19.687573 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.687577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.687660 (+ 83us) server_negotiation.cc:407] Connection header received
0811 22:30:19.687774 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.687778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.687861 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.687956 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.688374 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59826: BlockingRecv error: recv got EOF from 127.0.0.1:59826 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260811 22:30:19.700302 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.698920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59830 (local address 127.24.71.253:42681)
0811 22:30:19.699106 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.699113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.699273 (+ 160us) server_negotiation.cc:407] Connection header received
0811 22:30:19.699443 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.699446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.699495 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.699577 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.700169 (+ 592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59830: BlockingRecv error: recv got EOF from 127.0.0.1:59830 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 22:30:19.720400 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.719094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.24.71.253:42681)
0811 22:30:19.719273 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.719278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.719422 (+ 144us) server_negotiation.cc:407] Connection header received
0811 22:30:19.719543 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.719547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.719614 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.719733 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.720269 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59844: BlockingRecv error: recv got EOF from 127.0.0.1:59844 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 22:30:19.756069 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.754738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.24.71.253:42681)
0811 22:30:19.754897 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.754902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.755066 (+ 164us) server_negotiation.cc:407] Connection header received
0811 22:30:19.755206 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.755210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.755271 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.755382 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.755949 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59846: BlockingRecv error: recv got EOF from 127.0.0.1:59846 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 22:30:19.823997 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.822549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.24.71.253:42681)
0811 22:30:19.822726 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.822731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.822867 (+ 136us) server_negotiation.cc:407] Connection header received
0811 22:30:19.823045 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.823049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.823107 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.823201 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.823867 (+ 666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59858: BlockingRecv error: recv got EOF from 127.0.0.1:59858 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 22:30:19.955552 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:19.954230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59870 (local address 127.24.71.253:42681)
0811 22:30:19.954411 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:19.954416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:19.954572 (+ 156us) server_negotiation.cc:407] Connection header received
0811 22:30:19.954755 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:19.954758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:19.954816 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:19.954910 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:19.955414 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59870: BlockingRecv error: recv got EOF from 127.0.0.1:59870 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 22:30:20.215250 17548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:20.213933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.24.71.253:42681)
0811 22:30:20.214124 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:20.214128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:20.214146 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:30:20.214334 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:20.214338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:20.214405 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:20.214507 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:20.215114 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59880: BlockingRecv error: recv got EOF from 127.0.0.1:59880 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:30:20.405108 17563 catalog_manager.cc:1259] Loaded cluster ID: d318b925fad342a0a6c1b5b1fe8db59e
I20260811 22:30:20.405202 17563 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d: loading cluster ID for follower catalog manager: success
I20260811 22:30:20.407006 17563 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d: acquiring CA information for follower catalog manager: success
I20260811 22:30:20.408413 17563 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 903dee9da88d491a8c774e827bc2883d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:30:20.431466 17579 catalog_manager.cc:1259] Loaded cluster ID: d318b925fad342a0a6c1b5b1fe8db59e
I20260811 22:30:20.431561 17579 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652: loading cluster ID for follower catalog manager: success
I20260811 22:30:20.433338 17579 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652: acquiring CA information for follower catalog manager: success
I20260811 22:30:20.434778 17579 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4262e2126cd4413f89d8416bb1943652: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:30:20.733729 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:20.729526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.24.71.253:42681)
0811 22:30:20.729798 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:20.729804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:20.729911 (+ 107us) server_negotiation.cc:407] Connection header received
0811 22:30:20.730123 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:20.730126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:20.730268 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:20.730386 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:20.730661 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.731210 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.732143 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.732368 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.732495 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:20.732837 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:20.733288 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:20.733461 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:20.733500 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 22:30:20.733559 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":73,"threads_started":1}
I20260811 22:30:20.740628 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:20.736970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58266 (local address 127.24.71.252:41681)
0811 22:30:20.737186 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:20.737189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:20.737203 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:30:20.737286 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:20.737288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:20.737406 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:20.737508 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:20.737715 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.738260 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.739064 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.739278 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.739353 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:20.739690 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:20.740188 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:20.740295 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:20.740449 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 22:30:20.740503 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260811 22:30:20.749166 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:20.744168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.24.71.254:41177)
0811 22:30:20.744476 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:20.744482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:20.744498 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:30:20.744607 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:20.744610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:20.744801 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:20.744930 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:20.745301 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.746066 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.747092 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.747466 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.747575 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:20.748230 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:20.748683 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:20.748756 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:20.748916 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:30:20.748995 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":89,"threads_started":1}
I20260811 22:30:20.749356 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:20.744835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.24.71.252:41681)
0811 22:30:20.744994 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:20.745001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:20.745317 (+ 316us) server_negotiation.cc:407] Connection header received
0811 22:30:20.745532 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:20.745535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:20.745588 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:20.745666 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:20.745908 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.746635 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.747758 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.748138 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.748252 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:20.748603 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:20.748683 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:20.748764 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:20.749153 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 22:30:20.749217 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:30:20.749588 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:20.744493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.24.71.253:42681)
0811 22:30:20.744658 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:20.744663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:20.744785 (+ 122us) server_negotiation.cc:407] Connection header received
0811 22:30:20.745022 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:20.745027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:20.745109 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:20.745188 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:20.745529 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.746227 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.748402 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:20.748684 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:20.748792 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:20.749060 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:20.749152 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:20.749226 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:20.749399 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 22:30:20.749469 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:30:20.753368 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 17290
I20260811 22:30:20.763795 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 17374
I20260811 22:30:20.771739 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 17461
2026-08-11T22:30:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3525 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:30:20 dist-test-slave-mvvj krb5kdc[17708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:30:20 dist-test-slave-mvvj krb5kdc[17708](info): set up 2 sockets
Aug 11 22:30:20 dist-test-slave-mvvj krb5kdc[17708](info): commencing operation
krb5kdc: starting...
W20260811 22:30:21.888234 24863 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/test-user.keytab.
Aug 11 22:30:21 dist-test-slave-mvvj krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:30:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:30:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.71.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
Entry for principal kudu/127.24.71.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab.
I20260811 22:30:22.037669 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:37373
--webserver_interface=127.24.71.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:37373,127.24.71.253:42101,127.24.71.252:40075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:30:22.169060 17724 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:30:22.169438 17724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:22.169509 17724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:22.169603 17724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:22.173874 17724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:22.173974 17724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:22.174031 17724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:22.174084 17724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:22.174124 17724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:22.180570 17724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:37373,127.24.71.253:42101,127.24.71.252:40075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.254.keytab
--principal=kudu/127.24.71.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.254:37373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.71.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:22.181847 17724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:22.182969 17724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:22.192305 17732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:22.192296 17730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:22.192296 17729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:22.192376 17724 server_base.cc:1034] running on GCE node
I20260811 22:30:22.193281 17724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:22.194511 17724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:22.195683 17724 hybrid_clock.cc:648] HybridClock initialized: now 1786487422195665 us; error 35 us; skew 500 ppm
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:22.203703 17724 init.cc:377] Logged in from keytab as kudu/127.24.71.254@KRBTEST.COM (short username kudu)
I20260811 22:30:22.204829 17724 webserver.cc:460] Webserver started at http://127.24.71.254:41255/ using document root <none> and password file <none>
I20260811 22:30:22.205544 17724 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:22.205638 17724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:22.205883 17724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:22.208209 17724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/data/instance:
uuid: "e428371c61e64e0d934b2752895752a6"
format_stamp: "Formatted at 2026-08-11 22:30:22 on dist-test-slave-mvvj"
server_key: "70003b48435c6152d8f779242e898241"
server_key_iv: "2609f8a8b475cfe2f33439ad77a18b1c"
server_key_version: "encryptionkey@0"
I20260811 22:30:22.208827 17724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/wal/instance:
uuid: "e428371c61e64e0d934b2752895752a6"
format_stamp: "Formatted at 2026-08-11 22:30:22 on dist-test-slave-mvvj"
server_key: "70003b48435c6152d8f779242e898241"
server_key_iv: "2609f8a8b475cfe2f33439ad77a18b1c"
server_key_version: "encryptionkey@0"
I20260811 22:30:22.212855 17724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:30:22.215382 17741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:22.216732 17724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:30:22.216876 17724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/wal
uuid: "e428371c61e64e0d934b2752895752a6"
format_stamp: "Formatted at 2026-08-11 22:30:22 on dist-test-slave-mvvj"
server_key: "70003b48435c6152d8f779242e898241"
server_key_iv: "2609f8a8b475cfe2f33439ad77a18b1c"
server_key_version: "encryptionkey@0"
I20260811 22:30:22.216993 17724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:22.229717 17724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:22.233012 17724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:22.233229 17724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:22.240849 17724 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.254:37373
I20260811 22:30:22.241173 17798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.254:37373 every 8 connection(s)
I20260811 22:30:22.241902 17724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/data/info.pb
I20260811 22:30:22.244992 17799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:22.246338 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 17724
I20260811 22:30:22.246490 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-0/wal/instance
I20260811 22:30:22.246852 24863 external_mini_cluster.cc:1428] Setting key 5a2a116269764b78f2dd530e04a3a86b
I20260811 22:30:22.250341 17799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.71.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.253@KRBTEST.COM" created.
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:30:22.268317 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.252977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.24.71.254:37373)
0811 22:30:22.253451 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.253472 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.253750 (+ 278us) server_negotiation.cc:407] Connection header received
0811 22:30:22.254826 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.254830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.255026 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.255214 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.256182 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.257118 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.258146 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.258552 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.258737 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.261277 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.261283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.261294 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.261317 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.263373 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.265535 (+ 2162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.265541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.265542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.265664 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.266054 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.266056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.266057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.266496 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.266618 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.267149 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.267384 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":304,"thread_start_us":154,"threads_started":1}
I20260811 22:30:22.268317 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.252978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:37373 (local address 127.0.0.1:34014)
0811 22:30:22.253604 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.253642 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:22.254518 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:22.255213 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:22.255218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:22.255667 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:22.256029 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.256048 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.257247 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.257255 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.258011 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.258021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.258725 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.258729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.258843 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.259449 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:22.259467 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:22.261110 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:22.263521 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.263532 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.263543 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.263782 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.265770 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.265775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.265778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.265949 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.266626 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:22.266638 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:22.266842 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 22:30:22.267058 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 22:30:22.267459 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":41,"thread_start_us":178,"threads_started":1}
I20260811 22:30:22.270781 17799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:22.271672 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.271292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42101 (local address 127.0.0.1:50330)
0811 22:30:22.271419 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.271585 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:30:22.271896 17742 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
I20260811 22:30:22.272889 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.272673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42101 (local address 127.0.0.1:50342)
0811 22:30:22.272782 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.272829 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:30:22.273078 17799 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:42101: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab.
Entry for principal kudu/127.24.71.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab.
I20260811 22:30:22.280541 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.253:42101
--webserver_interface=127.24.71.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab
--principal=kudu/127.24.71.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:37373,127.24.71.253:42101,127.24.71.252:40075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:30:22.322309 17799 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } attempt: 1
I20260811 22:30:22.323274 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.322974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42101 (local address 127.0.0.1:50354)
0811 22:30:22.323128 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.323194 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 22:30:22.324064 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.323783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42101 (local address 127.0.0.1:50364)
0811 22:30:22.323937 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.323984 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 22:30:22.324193 17799 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:42101: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
I20260811 22:30:22.392344 17799 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } attempt: 2
I20260811 22:30:22.393163 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.392869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42101 (local address 127.0.0.1:50380)
0811 22:30:22.393011 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.393083 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 22:30:22.393905 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.393689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42101 (local address 127.0.0.1:50392)
0811 22:30:22.393798 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.393850 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 22:30:22.394033 17799 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.253:42101: Network error: Client connection negotiation failed: client connection to 127.24.71.253:42101: connect: Connection refused (error 111)
W20260811 22:30:22.397198 17807 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:30:22.397548 17807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:22.397606 17807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:22.397692 17807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:30:22.401997 17807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:22.402081 17807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:22.402120 17807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:22.402171 17807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:22.402211 17807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:22.407793 17807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:37373,127.24.71.253:42101,127.24.71.252:40075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.253.keytab
--principal=kudu/127.24.71.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.253:42101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.71.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:22.409061 17807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:22.410145 17807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:22.419269 17816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:22.419269 17814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:22.419275 17813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:22.419632 17807 server_base.cc:1034] running on GCE node
I20260811 22:30:22.420437 17807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:22.421706 17807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:22.422917 17807 hybrid_clock.cc:648] HybridClock initialized: now 1786487422422893 us; error 36 us; skew 500 ppm
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:22.425870 17807 init.cc:377] Logged in from keytab as kudu/127.24.71.253@KRBTEST.COM (short username kudu)
I20260811 22:30:22.426776 17807 webserver.cc:460] Webserver started at http://127.24.71.253:37685/ using document root <none> and password file <none>
I20260811 22:30:22.427366 17807 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:22.427454 17807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:22.427691 17807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:22.429844 17807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/data/instance:
uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a"
format_stamp: "Formatted at 2026-08-11 22:30:22 on dist-test-slave-mvvj"
server_key: "18cd1e8074b4454cbb9d12b701ce0df6"
server_key_iv: "dd4a9613eaf62d64cbdf1d3c10f6bf0f"
server_key_version: "encryptionkey@0"
I20260811 22:30:22.430419 17807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/wal/instance:
uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a"
format_stamp: "Formatted at 2026-08-11 22:30:22 on dist-test-slave-mvvj"
server_key: "18cd1e8074b4454cbb9d12b701ce0df6"
server_key_iv: "dd4a9613eaf62d64cbdf1d3c10f6bf0f"
server_key_version: "encryptionkey@0"
I20260811 22:30:22.434219 17807 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 22:30:22.436757 17823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:22.438108 17807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:30:22.438222 17807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/wal
uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a"
format_stamp: "Formatted at 2026-08-11 22:30:22 on dist-test-slave-mvvj"
server_key: "18cd1e8074b4454cbb9d12b701ce0df6"
server_key_iv: "dd4a9613eaf62d64cbdf1d3c10f6bf0f"
server_key_version: "encryptionkey@0"
I20260811 22:30:22.438325 17807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:22.450654 17807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:22.453786 17807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:22.453962 17807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:22.461035 17807 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.253:42101
I20260811 22:30:22.461329 17882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.253:42101 every 8 connection(s)
I20260811 22:30:22.462044 17807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/data/info.pb
I20260811 22:30:22.464954 17883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:22.468086 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 17807
I20260811 22:30:22.468228 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-1/wal/instance
I20260811 22:30:22.468590 24863 external_mini_cluster.cc:1428] Setting key 32e734aa5e9e6f6691b7389d2be427dc
I20260811 22:30:22.469019 17883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.71.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.71.252@KRBTEST.COM" created.
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:30:22.485193 17799 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } attempt: 3
I20260811 22:30:22.487334 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.471052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.24.71.254:37373)
0811 22:30:22.471217 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.471222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.472155 (+ 933us) server_negotiation.cc:407] Connection header received
0811 22:30:22.473605 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.473611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.473690 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.473775 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.475509 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.476107 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.477909 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.478139 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.478227 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.482783 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.482786 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.482788 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.482815 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.484736 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.485326 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.485329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.485335 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.485453 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.485868 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.485873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.485874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.486139 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.486222 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.486992 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.487140 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:30:22.488842 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.471437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:37373 (local address 127.0.0.1:34024)
0811 22:30:22.471986 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.472045 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:22.473335 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:22.473989 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:22.474001 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:22.474570 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:22.475326 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.475348 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.476260 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.476263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.477773 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.477789 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.478325 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.478332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.478573 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.479193 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:22.479210 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:22.482581 (+ 3371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:22.484884 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.484892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.484905 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.485182 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.485556 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.485561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.485563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.485743 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.486304 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:22.486311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:22.486637 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 22:30:22.487508 (+ 871us) client_negotiation.cc:240] Negotiation successful
0811 22:30:22.487870 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":115,"threads_started":1}
I20260811 22:30:22.490859 17883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
I20260811 22:30:22.498420 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.485661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42101 (local address 127.0.0.1:50404)
0811 22:30:22.485876 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.485893 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:22.485972 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:22.486757 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:22.486762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:22.486946 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:22.487183 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.487195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.488554 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.488559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.489463 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.489477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.490399 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.490402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.490483 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.491218 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:22.491230 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:22.494015 (+ 2785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:22.496394 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.496397 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.496399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.496674 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.496989 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.496992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.496995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.497100 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.497740 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:22.497745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:22.497877 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 22:30:22.498019 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 22:30:22.498188 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 22:30:22.498497 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.485784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.24.71.253:42101)
0811 22:30:22.486305 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.486322 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.486345 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:30:22.486383 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.486388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.486600 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.486746 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.487305 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.488422 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.489630 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.490129 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.490235 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.494144 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.494157 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.494174 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.494207 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.496240 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.496786 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.496795 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.496800 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.496893 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.497204 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.497208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.497212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.497622 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.497731 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.498131 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.498290 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.71.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab.
Entry for principal kudu/127.24.71.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab.
I20260811 22:30:22.499660 24863 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskapOnzC/build/debug/bin/kudu
/tmp/dist-test-taskapOnzC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.252:40075
--webserver_interface=127.24.71.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab
--principal=kudu/127.24.71.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.71.212:34881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.71.254:37373,127.24.71.253:42101,127.24.71.252:40075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:30:22.501685 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.491513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.24.71.253:42101)
0811 22:30:22.491876 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.491886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.491899 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:30:22.491931 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.491934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.491974 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.492083 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.492510 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.493134 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.494225 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.494458 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.494536 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.496839 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.496842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.496844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.496865 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.499722 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.500270 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.500276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.500280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.500394 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.500680 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.500686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.500690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.500908 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.501018 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.501309 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.501457 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":282,"thread_start_us":171,"threads_started":1}
I20260811 22:30:22.501948 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.491392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42101 (local address 127.0.0.1:50412)
0811 22:30:22.491498 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.491511 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:22.491583 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:22.492146 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:22.492149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:22.492303 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:22.492405 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.492411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.493280 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.493285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.494049 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.494060 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.494579 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.494586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.494710 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.495151 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:22.495172 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:22.496710 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:22.499868 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.499874 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.499879 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.500151 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.500500 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.500503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.500505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.500587 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.501017 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:22.501022 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:22.501127 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 22:30:22.501235 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:30:22.501806 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 22:30:22.501986 17799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:22.502786 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.502524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55452)
0811 22:30:22.502648 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.502708 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 22:30:22.502998 17883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:22.503765 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.503457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55476)
0811 22:30:22.503620 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.503704 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 22:30:22.504081 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.503295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55464)
0811 22:30:22.503700 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.503770 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":120,"threads_started":1}
W20260811 22:30:22.504060 17827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
W20260811 22:30:22.504227 17799 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:40075: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
I20260811 22:30:22.505013 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.504778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55490)
0811 22:30:22.504892 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.504939 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:30:22.505220 17883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:40075: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
I20260811 22:30:22.535409 17799 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } attempt: 1
I20260811 22:30:22.536514 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.536119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55502)
0811 22:30:22.536298 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.536385 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:30:22.537312 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.537076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55506)
0811 22:30:22.537190 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.537244 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:30:22.537456 17799 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:40075: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
I20260811 22:30:22.554486 17883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } attempt: 1
I20260811 22:30:22.555490 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.555116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55522)
0811 22:30:22.555319 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.555398 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:30:22.556654 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.556406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55528)
0811 22:30:22.556532 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.556592 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:30:22.556800 17883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:40075: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
I20260811 22:30:22.612625 17799 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } attempt: 2
I20260811 22:30:22.613605 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.613229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55544)
0811 22:30:22.613414 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.613495 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 22:30:22.614437 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.614164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55560)
0811 22:30:22.614298 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.614359 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:30:22.614645 17799 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:40075: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
W20260811 22:30:22.623059 17890 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:30:22.623394 17890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:30:22.623476 17890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:30:22.623577 17890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:30:22.624977 17883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } attempt: 2
I20260811 22:30:22.625869 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.625575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55568)
0811 22:30:22.625721 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.625787 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:30:22.626565 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.626351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55582)
0811 22:30:22.626456 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.626506 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 22:30:22.626680 17883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.71.252:40075: Network error: Client connection negotiation failed: client connection to 127.24.71.252:40075: connect: Connection refused (error 111)
W20260811 22:30:22.628772 17890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:30:22.628859 17890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:30:22.628907 17890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:30:22.628962 17890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:30:22.629001 17890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:30:22.635135 17890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.71.212:34881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.71.254:37373,127.24.71.253:42101,127.24.71.252:40075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/krb5kdc/kudu_127.24.71.252.keytab
--principal=kudu/127.24.71.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.71.252:40075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.71.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:22:59 UTC on ccycloud.quasar-ckgmnv.root.comops.site
build id 4581473
I20260811 22:30:22.636454 17890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:30:22.637610 17890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:30:22.646660 17901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:22.646659 17898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:30:22.646658 17899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:30:22.646889 17890 server_base.cc:1034] running on GCE node
I20260811 22:30:22.647601 17890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:30:22.648783 17890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:30:22.649969 17890 hybrid_clock.cc:648] HybridClock initialized: now 1786487422649952 us; error 30 us; skew 500 ppm
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:30:22.653138 17890 init.cc:377] Logged in from keytab as kudu/127.24.71.252@KRBTEST.COM (short username kudu)
I20260811 22:30:22.654170 17890 webserver.cc:460] Webserver started at http://127.24.71.252:42307/ using document root <none> and password file <none>
I20260811 22:30:22.654772 17890 fs_manager.cc:359] Metadata directory not provided
I20260811 22:30:22.654868 17890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:30:22.655109 17890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:30:22.657286 17890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/data/instance:
uuid: "8d728385e65249eab734a68e418cb8e1"
format_stamp: "Formatted at 2026-08-11 22:30:22 on dist-test-slave-mvvj"
server_key: "2387c914cda7ca1dd4d2cf683e9f7a02"
server_key_iv: "2efd9f8d0a70df4214f221f3ef8da03a"
server_key_version: "encryptionkey@0"
I20260811 22:30:22.657934 17890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/wal/instance:
uuid: "8d728385e65249eab734a68e418cb8e1"
format_stamp: "Formatted at 2026-08-11 22:30:22 on dist-test-slave-mvvj"
server_key: "2387c914cda7ca1dd4d2cf683e9f7a02"
server_key_iv: "2efd9f8d0a70df4214f221f3ef8da03a"
server_key_version: "encryptionkey@0"
I20260811 22:30:22.661854 17890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 22:30:22.664374 17909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:30:22.665725 17890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:30:22.665865 17890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/data,/tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/wal
uuid: "8d728385e65249eab734a68e418cb8e1"
format_stamp: "Formatted at 2026-08-11 22:30:22 on dist-test-slave-mvvj"
server_key: "2387c914cda7ca1dd4d2cf683e9f7a02"
server_key_iv: "2efd9f8d0a70df4214f221f3ef8da03a"
server_key_version: "encryptionkey@0"
I20260811 22:30:22.665980 17890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:30:22.681067 17890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:30:22.684182 17890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:30:22.684392 17890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:30:22.691789 17890 rpc_server.cc:307] RPC server started. Bound to: 127.24.71.252:40075
I20260811 22:30:22.692137 17967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.71.252:40075 every 8 connection(s)
I20260811 22:30:22.692910 17890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/data/info.pb
I20260811 22:30:22.696008 17968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:30:22.698379 24863 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskapOnzC/build/debug/bin/kudu as pid 17890
I20260811 22:30:22.698560 24863 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskapOnzC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786487121090521-24863-0/minicluster-data/master-2/wal/instance
I20260811 22:30:22.698860 24863 external_mini_cluster.cc:1428] Setting key 09ade33ee78de037fef8e54214b55028
I20260811 22:30:22.700364 17968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
I20260811 22:30:22.713096 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.702480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.24.71.254:37373)
0811 22:30:22.702645 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.702650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.702943 (+ 293us) server_negotiation.cc:407] Connection header received
0811 22:30:22.703099 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.703105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.703160 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.703252 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.704141 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.704716 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.705780 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.706011 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.706113 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.708851 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.708856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.708859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.708894 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.710657 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.711309 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.711315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.711320 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.711416 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.711856 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.711859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.711861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.712109 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.712185 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.712770 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.712918 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:30:22.713842 17799 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } attempt: 3
I20260811 22:30:22.713959 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.702401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.254:37373 (local address 127.0.0.1:34040)
0811 22:30:22.702808 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.702858 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:22.702982 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:22.703311 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:22.703318 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:22.703687 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:22.704001 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.704014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.704823 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.704827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.705652 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.705662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.706169 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.706174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.706325 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.707115 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:22.707131 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:22.708674 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:22.710786 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.710795 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.710804 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.711205 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.711524 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.711528 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.711529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.711727 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.712218 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:22.712228 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:22.712514 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 22:30:22.712709 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 22:30:22.713058 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":51,"spinlock_wait_cycles":768,"thread_start_us":169,"threads_started":1}
I20260811 22:30:22.713973 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.700354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55584 (local address 127.24.71.252:40075)
0811 22:30:22.700944 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.700965 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.701063 (+ 98us) server_negotiation.cc:407] Connection header received
0811 22:30:22.701927 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.701950 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.702364 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.702723 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.703929 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.704889 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.705789 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.706171 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.706330 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.709303 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.709314 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.709330 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.709375 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.711277 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.711811 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.711819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.711820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.711932 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.712261 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.712264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.712265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.712521 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.712694 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.712970 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.713125 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":377,"thread_start_us":149,"threads_started":1}
I20260811 22:30:22.715521 17968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:22.717819 17883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } attempt: 3
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.254@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.253@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.254@KRBTEST.COM
I20260811 22:30:22.724800 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.714429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55596)
0811 22:30:22.714550 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.714566 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:22.714649 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:22.714947 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:22.714951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:22.715160 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:22.715281 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.715289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.716382 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.716386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.717178 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.717190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.718144 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.718151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.718261 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.718812 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:22.718827 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:22.720582 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:22.722865 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.722871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.722876 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.723161 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.723528 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.723535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.723539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.723645 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.724192 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:22.724196 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:22.724299 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 22:30:22.724418 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:30:22.724593 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 22:30:22.725498 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.714553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.24.71.252:40075)
0811 22:30:22.714661 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.714665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.714702 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:30:22.714737 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.714740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.714812 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.715398 (+ 586us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.715506 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.716253 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.717332 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.717712 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.718527 (+ 815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.720790 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.720794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.720796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.720822 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.722724 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.723308 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.723313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.723317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.723405 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.723794 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.723799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.723802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.724071 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.724191 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.725168 (+ 977us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.725294 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:30:22.727999 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.716167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.24.71.253:42101)
0811 22:30:22.716286 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.716291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.716306 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:30:22.716379 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.716382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.716425 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.716519 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.717206 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.717844 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.718947 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.719324 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.719426 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.723491 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.723497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.723502 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.723532 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.725311 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.726304 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.726311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.726313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.726400 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.726828 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.726834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.726838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.727133 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.727214 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.727603 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.727727 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":51}
I20260811 22:30:22.728711 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.716053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.253:42101 (local address 127.0.0.1:50414)
0811 22:30:22.716203 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.716217 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:22.716289 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:22.716744 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:22.716747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:22.716918 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:22.717042 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.717050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.717978 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.717984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.718810 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.718822 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.719569 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.719575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.719675 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.720910 (+ 1235us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:22.720925 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:22.722812 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:22.725422 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.725428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.725429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.726174 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.726596 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.726600 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.726604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.726701 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.727297 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:22.727300 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:22.727400 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 22:30:22.728393 (+ 993us) client_negotiation.cc:240] Negotiation successful
0811 22:30:22.728538 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":5}
I20260811 22:30:22.729130 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.718218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55612)
0811 22:30:22.718445 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.718457 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:22.718534 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:22.719182 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:22.719185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:22.719367 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:22.719481 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.719488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.720808 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.720812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.721646 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.721658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.722454 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.722457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.722541 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.722977 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:22.722991 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:22.725517 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:22.727439 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.727442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.727444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.727706 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.728082 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.728085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.728087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.728183 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.728632 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:22.728635 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:22.728727 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 22:30:22.728821 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:30:22.728949 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":3}
I20260811 22:30:22.729305 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.716985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34050 (local address 127.24.71.254:37373)
0811 22:30:22.717093 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.717099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.717185 (+ 86us) server_negotiation.cc:407] Connection header received
0811 22:30:22.717310 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.717313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.717354 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.717430 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.718213 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.719266 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.720721 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.721406 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.721494 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.725199 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.725203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.725205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.725229 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.727305 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.727867 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.727871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.727873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.727964 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.728317 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.728323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.728324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.728540 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.728682 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.729052 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.729168 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:30:22.729578 17799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6: Bootstrap starting.
I20260811 22:30:22.729884 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.718652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55612 (local address 127.24.71.252:40075)
0811 22:30:22.718970 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.718980 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.718992 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:30:22.719022 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.719025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.719061 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.719593 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.719673 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.720666 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.721831 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.722301 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.722446 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.725675 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.725680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.725684 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.725709 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.727327 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.727870 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.727873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.727875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.727955 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.728318 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.728321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.728323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.728516 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.729505 (+ 989us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.729580 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.729710 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260811 22:30:22.732556 17799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:22.732741 17968 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:30:22.733521 17799 log.cc:824] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:22.735029 17883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a: Bootstrap starting.
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.71.253@KRBTEST.COM
I20260811 22:30:22.737586 17883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a: Neither blocks nor log segments found. Creating new log.
Aug 11 22:30:22 dist-test-slave-mvvj krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786487422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.71.252@KRBTEST.COM for kudu/127.24.71.252@KRBTEST.COM
I20260811 22:30:22.738364 17883 log.cc:824] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:22.740514 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.730835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50426 (local address 127.24.71.253:42101)
0811 22:30:22.730959 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.730963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.730978 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:30:22.731017 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.731022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.731064 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.731152 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.731680 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.732441 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.733476 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.733824 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.733904 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.736743 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.736747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.736750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.736776 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.738539 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.739035 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.739041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.739046 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.739130 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.739464 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.739467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.739469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.739679 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.739862 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.740249 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.740362 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:30:22.740815 17799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6: No bootstrap required, opened a new log
I20260811 22:30:22.740819 17883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a: No bootstrap required, opened a new log
I20260811 22:30:22.742811 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.733176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.71.252:40075 (local address 127.0.0.1:55616)
0811 22:30:22.733438 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:30:22.733454 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:30:22.733521 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:30:22.733849 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:30:22.733855 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:30:22.734050 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:30:22.734134 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.734140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.735162 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.735167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.736039 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:30:22.736053 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.736639 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.736646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:30:22.736743 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.737190 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:30:22.737201 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:30:22.738702 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:30:22.741292 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.741295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.741296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.741557 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.741841 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:30:22.741844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:30:22.741846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:30:22.741924 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:30:22.742338 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:30:22.742341 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:30:22.742411 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 22:30:22.742501 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:30:22.742639 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260811 22:30:22.743335 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.733333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55616 (local address 127.24.71.252:40075)
0811 22:30:22.733638 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.733642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.733650 (+ 8us) server_negotiation.cc:407] Connection header received
0811 22:30:22.733674 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.733676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.733707 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.733763 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.734267 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.735019 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.736168 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.736509 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.736615 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.738836 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.738841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.738846 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.738873 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.741173 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.741663 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.741666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.741668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.741747 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.742029 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.742032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.742034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.742223 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.742307 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.743077 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.743201 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260811 22:30:22.744489 24863 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:30:22.745309 17799 raft_consensus.cc:348] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } }
I20260811 22:30:22.745311 17883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } }
I20260811 22:30:22.745553 17799 raft_consensus.cc:729] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e428371c61e64e0d934b2752895752a6, State: Initialized, Role: FOLLOWER
I20260811 22:30:22.745581 17883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63444dd3c60d4ba69fdaaf3e5ff16c1a, State: Initialized, Role: FOLLOWER
I20260811 22:30:22.745735 17769 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e428371c61e64e0d934b2752895752a6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34050
I20260811 22:30:22.745908 17883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } }
I20260811 22:30:22.745909 17799 consensus_queue.cc:260] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } }
I20260811 22:30:22.746555 17968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1: Bootstrap starting.
I20260811 22:30:22.746460 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.746649 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:22.747330 17799 sys_catalog.cc:564] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:22.747340 17883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:22.747386 17769 raft_consensus.cc:482] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:30:22.747488 17769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:22.749641 17968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1: Neither blocks nor log segments found. Creating new log.
I20260811 22:30:22.749598 17769 raft_consensus.cc:504] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } }
I20260811 22:30:22.750651 17968 log.cc:824] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:30:22.751232 17854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e428371c61e64e0d934b2752895752a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a"
I20260811 22:30:22.751468 17854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:22.752872 17769 leader_election.cc:288] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [CANDIDATE]: Term 1 election: Requested vote from peers 63444dd3c60d4ba69fdaaf3e5ff16c1a (127.24.71.253:42101), 8d728385e65249eab734a68e418cb8e1 (127.24.71.252:40075)
I20260811 22:30:22.755739 17854 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e428371c61e64e0d934b2752895752a6 in term 1.
W20260811 22:30:22.756129 17998 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:22.756217 17998 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:22.756537 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.756691 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:22.757668 17742 leader_election.cc:302] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63444dd3c60d4ba69fdaaf3e5ff16c1a, e428371c61e64e0d934b2752895752a6; no voters:
I20260811 22:30:22.758174 17977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:30:22.759173 17977 raft_consensus.cc:686] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [term 1 LEADER]: Becoming Leader. State: Replica: e428371c61e64e0d934b2752895752a6, State: Running, Role: LEADER
I20260811 22:30:22.759531 17977 consensus_queue.cc:237] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } }
W20260811 22:30:22.760237 18004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:22.760321 18004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:22.761406 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e428371c61e64e0d934b2752895752a6. Latest consensus state: current_term: 1 leader_uuid: "e428371c61e64e0d934b2752895752a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.761535 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:22.759078 17807 master_runner.cc:186] Error getting master registration for 127.24.71.252:40075: OK, instance_id { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" instance_seqno: 1786487422689752 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:30:22.762436 17807 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:30:22.764926 17968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1: No bootstrap required, opened a new log
I20260811 22:30:22.763782 17724 master_runner.cc:186] Error getting master registration for 127.24.71.252:40075: OK, instance_id { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" instance_seqno: 1786487422689752 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:30:22.767186 17724 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:30:22.769181 17968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } }
I20260811 22:30:22.769474 17968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d728385e65249eab734a68e418cb8e1, State: Initialized, Role: FOLLOWER
I20260811 22:30:22.769847 17968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } }
I20260811 22:30:22.771406 17968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:30:22.771960 17939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e428371c61e64e0d934b2752895752a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8d728385e65249eab734a68e418cb8e1"
I20260811 22:30:22.772217 17939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:30:22.773742 17939 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e428371c61e64e0d934b2752895752a6 in term 1.
I20260811 22:30:22.781351 18018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:30:22.782390 18018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:30:22.784340 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.755077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.24.71.254:37373)
0811 22:30:22.755198 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.755203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.769089 (+ 13886us) server_negotiation.cc:407] Connection header received
0811 22:30:22.769233 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.769236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.769297 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.769394 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.770879 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.771702 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.777704 (+ 6002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.778056 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.778157 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.780062 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.780067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.780070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.780099 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.782414 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.782955 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.782958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.782961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.783045 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.783329 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.783332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.783334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.783553 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.783666 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.784056 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.784183 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:30:22.790134 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.755386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55620 (local address 127.24.71.252:40075)
0811 22:30:22.755542 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.755548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.777932 (+ 22384us) server_negotiation.cc:407] Connection header received
0811 22:30:22.778115 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.778119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.778169 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.778225 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.779263 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.779979 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.781120 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.781475 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.781589 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.786450 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.786454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.786458 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.786497 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.788225 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.788747 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.788751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.788753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.788846 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.789150 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.789155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.789157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.789367 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.789473 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.789843 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.789972 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 22:30:22.792021 18033 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:30:22.792109 18033 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:30:22.793499 18021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.793730 18021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:22.797995 17939 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [term 1 FOLLOWER]: Refusing update from remote peer e428371c61e64e0d934b2752895752a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:22.798709 18016 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:22.799711 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:22.762042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.24.71.253:42101)
0811 22:30:22.762149 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.762154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.784209 (+ 22055us) server_negotiation.cc:407] Connection header received
0811 22:30:22.784388 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.784391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.784444 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.784493 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.785358 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.786130 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.793362 (+ 7232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:22.793728 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:22.793848 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:22.795721 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:30:22.795726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:30:22.795729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:30:22.795760 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:30:22.797527 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.798061 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.798065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.798067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.798154 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:30:22.798567 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:30:22.798570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:30:22.798572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:30:22.798839 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:30:22.798944 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:22.799420 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 22:30:22.799547 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:30:22.803779 18021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e428371c61e64e0d934b2752895752a6. Latest consensus state: current_term: 1 leader_uuid: "e428371c61e64e0d934b2752895752a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.803937 18021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:22.805018 17854 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [term 1 FOLLOWER]: Refusing update from remote peer e428371c61e64e0d934b2752895752a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:30:22.805697 18016 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:30:22.807102 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e428371c61e64e0d934b2752895752a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.807226 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:22.807700 18035 mvcc.cc:205] Tried to move back new op lower bound from 7317452483772162048 to 7317452483626823680. Current Snapshot: MvccSnapshot[applied={T|T < 7317452483772162048}]
I20260811 22:30:22.810302 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e428371c61e64e0d934b2752895752a6. Latest consensus state: current_term: 1 leader_uuid: "e428371c61e64e0d934b2752895752a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.810439 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:22.810762 18018 catalog_manager.cc:1347] Generated new cluster ID: bbdcefaa33544e3589d13eb5141e9d10
I20260811 22:30:22.810850 18018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:30:22.816264 18016 sys_catalog.cc:455] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e428371c61e64e0d934b2752895752a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.816408 18016 sys_catalog.cc:458] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:30:22.816890 18021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e428371c61e64e0d934b2752895752a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.817018 18021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:22.817179 18036 mvcc.cc:205] Tried to move back new op lower bound from 7317452483772162048 to 7317452483626823680. Current Snapshot: MvccSnapshot[applied={T|T < 7317452483772162048}]
I20260811 22:30:22.818986 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e428371c61e64e0d934b2752895752a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e428371c61e64e0d934b2752895752a6" member_type: VOTER last_known_addr { host: "127.24.71.254" port: 37373 } } peers { permanent_uuid: "63444dd3c60d4ba69fdaaf3e5ff16c1a" member_type: VOTER last_known_addr { host: "127.24.71.253" port: 42101 } } peers { permanent_uuid: "8d728385e65249eab734a68e418cb8e1" member_type: VOTER last_known_addr { host: "127.24.71.252" port: 40075 } } }
I20260811 22:30:22.819118 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:30:22.819551 18040 mvcc.cc:205] Tried to move back new op lower bound from 7317452483772162048 to 7317452483626823680. Current Snapshot: MvccSnapshot[applied={T|T < 7317452483772162048}]
I20260811 22:30:22.839696 18018 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:30:22.841331 18018 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:30:22.864539 18018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e428371c61e64e0d934b2752895752a6: Generated new TSK 0
I20260811 22:30:22.865566 18018 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:30:22.905953 17973 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:22.903172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55622 (local address 127.24.71.252:40075)
0811 22:30:22.903311 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.903315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.904440 (+ 1125us) server_negotiation.cc:407] Connection header received
0811 22:30:22.904773 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.904781 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.904887 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.904983 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.905789 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55622: BlockingRecv error: recv got EOF from 127.0.0.1:55622 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 22:30:22.907683 17802 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:22.903725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.24.71.254:37373)
0811 22:30:22.903888 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.903893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.906679 (+ 2786us) server_negotiation.cc:407] Connection header received
0811 22:30:22.906862 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.906866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.906920 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.906984 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.907547 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34076: BlockingRecv error: recv got EOF from 127.0.0.1:34076 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 22:30:22.909806 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:22.908440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.24.71.253:42101)
0811 22:30:22.908571 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.908575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.908750 (+ 175us) server_negotiation.cc:407] Connection header received
0811 22:30:22.909047 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.909051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.909104 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.909157 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.909678 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50446: BlockingRecv error: recv got EOF from 127.0.0.1:50446 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 22:30:22.914969 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:22.913292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.24.71.253:42101)
0811 22:30:22.913403 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.913407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.913864 (+ 457us) server_negotiation.cc:407] Connection header received
0811 22:30:22.914132 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.914136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.914217 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.914272 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.914841 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50460: BlockingRecv error: recv got EOF from 127.0.0.1:50460 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 22:30:22.921542 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:22.919477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50466 (local address 127.24.71.253:42101)
0811 22:30:22.919610 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.919614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.920388 (+ 774us) server_negotiation.cc:407] Connection header received
0811 22:30:22.920603 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.920606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.920664 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.920716 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.921417 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50466: BlockingRecv error: recv got EOF from 127.0.0.1:50466 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 22:30:22.928259 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:22.926373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50482 (local address 127.24.71.253:42101)
0811 22:30:22.926506 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.926511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.927119 (+ 608us) server_negotiation.cc:407] Connection header received
0811 22:30:22.927345 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.927348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.927409 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.927471 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.928130 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50482: BlockingRecv error: recv got EOF from 127.0.0.1:50482 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 22:30:22.937366 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:22.935238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.24.71.253:42101)
0811 22:30:22.935426 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.935431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.936072 (+ 641us) server_negotiation.cc:407] Connection header received
0811 22:30:22.936467 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.936470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.936538 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.936608 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.937244 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50498: BlockingRecv error: recv got EOF from 127.0.0.1:50498 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 22:30:22.950431 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:22.948430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50512 (local address 127.24.71.253:42101)
0811 22:30:22.948559 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.948564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.949276 (+ 712us) server_negotiation.cc:407] Connection header received
0811 22:30:22.949494 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.949498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.949575 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.949630 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.950302 (+ 672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50512: BlockingRecv error: recv got EOF from 127.0.0.1:50512 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 22:30:22.970991 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:22.969493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50520 (local address 127.24.71.253:42101)
0811 22:30:22.969633 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:22.969637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:22.969950 (+ 313us) server_negotiation.cc:407] Connection header received
0811 22:30:22.970166 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:22.970170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:22.970227 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:22.970319 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:22.970861 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50520: BlockingRecv error: recv got EOF from 127.0.0.1:50520 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 22:30:23.006666 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:23.005193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.24.71.253:42101)
0811 22:30:23.005338 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:23.005343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:23.005627 (+ 284us) server_negotiation.cc:407] Connection header received
0811 22:30:23.005819 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:23.005823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:23.005888 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:23.005984 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:23.006532 (+ 548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50536: BlockingRecv error: recv got EOF from 127.0.0.1:50536 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 22:30:23.074311 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:23.072982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50548 (local address 127.24.71.253:42101)
0811 22:30:23.073174 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:23.073178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:23.073290 (+ 112us) server_negotiation.cc:407] Connection header received
0811 22:30:23.073494 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:23.073497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:23.073548 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:23.073626 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:23.074169 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50548: BlockingRecv error: recv got EOF from 127.0.0.1:50548 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 22:30:23.206163 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:23.204736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.24.71.253:42101)
0811 22:30:23.204950 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:23.204955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:23.205070 (+ 115us) server_negotiation.cc:407] Connection header received
0811 22:30:23.205264 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:23.205268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:23.205328 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:23.205434 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:23.206017 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50564: BlockingRecv error: recv got EOF from 127.0.0.1:50564 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 22:30:23.471908 17889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:30:23.464784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50578 (local address 127.24.71.253:42101)
0811 22:30:23.465031 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:23.465036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:23.470924 (+ 5888us) server_negotiation.cc:407] Connection header received
0811 22:30:23.471079 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:23.471081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:23.471137 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:23.471224 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:30:23.471769 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50578: BlockingRecv error: recv got EOF from 127.0.0.1:50578 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 22:30:23.761684 18004 catalog_manager.cc:1259] Loaded cluster ID: bbdcefaa33544e3589d13eb5141e9d10
I20260811 22:30:23.761806 18004 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a: loading cluster ID for follower catalog manager: success
I20260811 22:30:23.763553 18004 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a: acquiring CA information for follower catalog manager: success
I20260811 22:30:23.764796 18004 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 63444dd3c60d4ba69fdaaf3e5ff16c1a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:30:23.793480 18033 catalog_manager.cc:1259] Loaded cluster ID: bbdcefaa33544e3589d13eb5141e9d10
I20260811 22:30:23.793576 18033 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1: loading cluster ID for follower catalog manager: success
I20260811 22:30:23.795243 18033 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1: acquiring CA information for follower catalog manager: success
I20260811 22:30:23.796487 18033 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8d728385e65249eab734a68e418cb8e1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:30:23.990556 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:23.986257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.24.71.253:42101)
0811 22:30:23.986511 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:23.986516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:23.986629 (+ 113us) server_negotiation.cc:407] Connection header received
0811 22:30:23.986809 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:23.986813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:23.986972 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:23.987103 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:23.987375 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:23.987981 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:23.988885 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:23.989125 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:23.989232 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:23.989580 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:23.990058 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:23.990196 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:23.990287 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 22:30:23.990373 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":70,"threads_started":1}
I20260811 22:30:23.997819 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:23.993908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55634 (local address 127.24.71.252:40075)
0811 22:30:23.994172 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:23.994176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:23.994189 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:30:23.994275 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:23.994277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:23.994417 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:23.994522 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:23.994747 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:23.995473 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:23.996367 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:23.996593 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:23.996716 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:23.997009 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:23.997438 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:23.997546 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:23.997642 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 22:30:23.997694 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":120,"threads_started":1}
I20260811 22:30:24.005505 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:24.001268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55636 (local address 127.24.71.252:40075)
0811 22:30:24.001394 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:24.001399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:24.001790 (+ 391us) server_negotiation.cc:407] Connection header received
0811 22:30:24.002040 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:24.002043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:24.002091 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:24.002161 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:24.002508 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:24.003308 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:24.004362 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:24.004644 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:24.004738 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:24.005026 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:24.005111 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:24.005201 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:24.005334 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 22:30:24.005389 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:30:24.005494 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:24.001417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.24.71.253:42101)
0811 22:30:24.001536 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:24.001540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:24.001713 (+ 173us) server_negotiation.cc:407] Connection header received
0811 22:30:24.001963 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:24.001968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:24.002042 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:24.002126 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:24.002363 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:24.002953 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:24.004132 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:24.004477 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:24.004609 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:24.004908 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:24.005024 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:24.005099 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:24.005269 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:30:24.005344 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 22:30:24.006438 18130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:30:24.001131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34078 (local address 127.24.71.254:37373)
0811 22:30:24.001636 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 22:30:24.001642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:30:24.001661 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:30:24.001836 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:30:24.001840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:30:24.002049 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:30:24.002173 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:30:24.002566 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:24.003395 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:24.004660 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:30:24.005051 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:30:24.005192 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:30:24.005638 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:30:24.006021 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:30:24.006123 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:30:24.006210 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 22:30:24.006279 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":110,"threads_started":1}
I20260811 22:30:24.010069 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 17724
I20260811 22:30:24.019825 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 17807
I20260811 22:30:24.028064 24863 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskapOnzC/build/debug/bin/kudu with pid 17890
2026-08-11T22:30:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3252 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6778 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (302916 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST